Everyday Research Methods

Claim: Mothers' friendships are good for babies' brains

The study showed that moms with larger social support networks had kids with higher cognitive test scores. Photo: Noriko Cooper / Alamy Stock Photo

The  Washington Post headline reads, "Mothers' friendships may be good for babies' brains."  The story is about an empirical study on mothers' social networks. Although it says "babies' brains", the study measured the babies' cognitive skill (there were no baby brain scans). The empirical article came out in JAMA Network Open. Does the original study support the claim? Let's find out.Before reading on, answer these questions:

 

a) What are the two variables in the Washington Post headline claim? What makes the claim a causal one? 

b) Of course, we need to conduct an experiment to definitively support a causal claim. Would an experiment be possible in this particular case?  

Here are some details about the study, as summarized by the journalist (not the researcher):

For the current study, researchers examined data on 1,082 mother-child pairs. They questioned women about their family structure, friendships and relationships in their communities and also looked at test results from cognitive assessments done when children were 2 years old.

Overall, mothers had an average of 3.5 friends in their social support networks. The kids of mothers with more than that tended to have higher cognitive test scores than the kids of those who had fewer, suggesting “network conditions were significantly associated with early cognitive development in children,” the study authors wrote.

c) Sketch a  simple scatterplot of the result described above. What will you put on the two axes? How many dots should your scatterplot have? What does one dot represent? 

Read the following details if you've studied Chapter 9: 

The researchers accounted for some factors that might influence child cognitive development, including mother’s age and IQ, father’s education level, and the child’s birth weight. 

In addition, "researchers also accounted for poverty levels"

d) The paragraph above is describing a multiple regression analysis. What would be the dependent (criterion variable) in this analysis? What are the predictor variables? (Hint: There are six predictor variables).

e) Imagine that someone proposes income as a possible internal validity problem. Given what you know about the study, can we say that "Perhaps friendship network size and child cognitive skills are correlated because of the third variable of income: That is, higher-income mothers may have more friends and more cognitively skilled children"?  Would this criticism be reasonable? Why or why not?

f) The journalist writes: "...they did not have data to assess other personal differences, such as history of depression, between mothers with large networks of friends and those with fewer."
This means history of depression could be a possible third variable explaining the relationship. How could the researchers rule out this third variable? 

The journalist writes: 

All of the families in the study lived in the area around Memphis. It is possible that results might be different elsewhere.

g) Which big validity is the journalist addressing in the statement above? 

h) Do you think the correlation between friendship and cognitive test scores would actually come out differently in a different city or country? Why or why not? 

 

Suggested answers:

a) The two variables are Amount of mothers' friendships and Level of health of babies' brains. The language of "may be good for" makes this a causal headline (remember we ignore adverbs such as "may" when classifying claims). 

b) Probably not; you'd have to randomly assign some mothers to have more friends and others to have fewer friends.

c)  One axis should have "number of friends" and the other should have "cognitive test score". There should be about 1082 dots on the scatterplot (better count your dots now...), and each dot represents a mother-child dyad.

d) The criterion variable is cognitive test score. The predictor variables are number of friends, poverty levels, mother’s age, mother's IQ, father’s education level and the child’s birth weight. 

e) Since the researchers did control for poverty levels, we can actually rule out wealth as a possible third variable. 

f) the researchers could obtain history of depression data on each mother and then add this variable as a predictor variable to the regression analysis. If the beta associated with "number of friends" is still significantly predicting cognitive test score, then we can rule out this possible confound (/third variable/ internal validity problem).

g) external validity

h) In my opinion, it seems unlikely that such a relationship is limited only to Memphis residents. 

Note: You can read the original empirical article in JAMA, here. Note to savvy instructors: The p-curve on the results in this paper suggests that there may be little evidentiary value. 

Which comes first? Cross-lag study on depression and social media use

Which comes first, social media use or depression? Photo:Africa Studio/Shutterstock

We have seen multiple studies that have tried to estimate how strong the link is between social media use and depression among young adults. We've covered some of these studies on this blog, including the effect size of the correlation and even a brain imaging study that manipulated "likes."

One challenge with studying real-world links between social media use and depression is that most use correlational designs. Therefore, we can't be sure whether any link between social media use and depression means that social media leads to more depression, or if depression leads to more social media use. And of course, in correlational studies there may even be a third variable, such as age, which is associated with both variables and causing them to appear correlated. 

The latest study used a longitudinal, cross-lag panel design, such as those described in Chapter 9, to study this question. Here's a writeup of the study by a journalist at Earth.com.

First, here are some comments about the design: 

The researchers point out that previous studies that found links between depression and social media use among tweens are limited in their results because they only captured a snapshot of teen social media use.

“You have to follow the same people over time in order to draw the conclusion that social media use predicts greater depressive symptoms,” said Taylor Heffer, the lead author of the study. “By using two large longitudinal samples, we were able to empirically test that assumption.”

Here are some more details about the method. They actually used two longitudinal samples: 

For this study, the researchers conducted a two-year longitudinal study surveying 6th, 7th, and 8th graders in Canada. The team also surveyed a series of undergraduate students once a year for six years as they progressed through college.

What were the results? 

Higher depression symptoms predicted social media use among teen girls which indicates that unhealthy social media use may be a symptom depression, not a catalyst for it.

“This finding contrasts with the idea that people who use a lot of social media become more depressed over time. Instead, adolescent girls who are feeling down may turn to social media to try and make themselves feel better,” said Heffer.

Questions

a) In a cross-lag panel design, researchers measure the same two variables in the same sample of people over time. What are the two variables they focused on here?

b) Sketch a series of boxes depicting the design of the study, similar to the figures in Chapter 9's cross-lag panel section. How many time periods would you need to depict for the undergraduate sample? 

c) Focus only on the girls. What do you think the autocorrelations might have been in this design? Fill those in on your sketch.

d) Still focusing only on the girls, given the description of the results, estimate what the two cross-lag correlations might have been. Which correlation should be larger--the one from Time 1 depression to Time 2 social media, or the one from Time 1 social media to Time 2 depression?

 

Do kids do better in private schools?

What evidence would it take to convince us that it's the schools, not other factors, that are responsible for the outcomes of private school students? Photo: Image Source / Alamy Stock Photo

A large study has compared the outcomes of children who've attended private schools to those who've attended public schools.  A journalist summarized the report in the Washington Post. The study provides a nice example of how multivariate regression can be used to test third variable hypotheses. 

When we look simply at the educational acheivement of students in private schools vs. public schools, private school students have higher achievement scores. However, all such studies are correlational, because the two variables--Type of School and Level of Achievement--are measured. 

Therefore, such studies how covariance, because the results depict a relationship. The study may even show temporal precedence, because attending school presumably precedes the measure of achievement. However, such studies are weak on internal validity. We can think of several alternative explanations for why children in private schools are scoring higher. 

One major alternative explanation is socioeconomic status. Children from wealthier families are more likely to afford private schools. And in general, children from wealthier families tend to score higher on achievement tests. 

The Washington Post journalist quoted one of study's authors, Robert Pianta, who summed up the study's results this way:

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Questions

a) Draw little diagrams similar to those in Figure 8.15 (in the 3rd ed.) to depict the arguments being made in this study. What would A be? What about B? In the quote from Pianta, above, what would the C variable(s) be? 

b) The researchers used type of school (private vs. public), which is a categorical variable. But in some analyses, the researchers also used "number of years in private school" as an alternative version of this variable. Is "number of years in private school" categorical, ordinal, interval, or ratio data? 

c) Sketch a mock-up regression table with the criterion variable  at the top and predictors below (Use Table 9.1 as a model). Which variable do you think the researchers selected as the criterion (dependent) variable in their analyses? Which variable(s) would have been the predictors? 

d) Now that you know what the results were, think about how the beta associated with "number of years in private school"  would change when parental SES is added and removed from the regression analyses. 

Original journal article seems to be open-access. Give it a try!

Suggested answers

a) A and B would be Type of School and Level of Acheivement. It doesn't really  matter which one is A and which one is called B. C would be Family Income and/or Parental Education.

b) Ratio data (zero is meaningful in this scale because you could attend zero years of private school)

c) The criterion variable would be Achievement, and the predictors would be Number of Years of Private School, Family Income, and Parental Education.

d)When the Number of Years of Private School is on the table (in the analysis) by itself, its beta is likely to be positve and significant (more years of private school goes with higher achievement). When Family Income and Parental Education are added to the table, the beta for Number of Years of Private School should drop to zero.  This pattern of results is consistent with the argument that Family Income and Parental Education are the alternative explanation for the original relationship .

Delay of gratification (marshmallow study replication)

Her school achievement later in life can be predicted from her ability to wait for a treat (or by her family's SES). Photo: Manley099/Getty Images

There's a new replication study about the famous "marshmallow study",  and it's all over the popular press. You've probably heard of the original research: Kids are asked to sit alone in a room with a single marshmallow (or some other treat they like, such as pretzels). If the child can wait for up to 15 minutes until the experimenter comes back, they receive two marshmallows. But if they eat the first one early, they don't. As part of the original study, kids were tracked over several years. One of the key findings was that the longer children were able to wait at age 4, the better they were doing in school as teenagers. Psychologists have often used this study as an illustration of how self-control is related to important life outcomes. 

The press coverage of this year's replication study illustrates at least two things. First, it's a nice example of multiple regression.  Second, it's an example of how different media outlets assign catchy--but sometimes erroneous--headlines on the same study.

First, let's talk about the multiple regression piece. Regression analyses often try to understand a core bivariate relationship more fully. In this case, the core relationship they start with is between the two variables, "length of time kids waited at age 4" and "test performance at age 15." Here's how it was described by Payne and Sheeran in the online magazine Behavioral Scientist:

The result? Kids who resisted temptation longer on the marshmallow test had higher achievement later in life. The correlation was in the same direction as in Mischel’s early study. It was statistically significant, like the original study. The correlation was somewhat smaller, and this smaller association is probably the more accurate estimate, because the sample size in the new study was larger than the original. Still, this finding says that observing a child for seven minutes with candy can tell you something remarkable about how well the child is likely to do in high school. 

a) Sketch a well-labelled scatterplot of the relationship described above. What direction will the dots slope? Will they be fairly tight to a straight line, or spread out?

b) The writers (Payne and Sheeran) suggest that a larger sample size leads to a more accurate estimate of a correlation. Can you explain why a large sample size might give a more accurate statistical estimate? (Hint: Chapter 8 talks about outliers and sample size--see Figures 8.10 and 8.11.)

Now here's more about the study: 

The researchers next added a series of “control variables” using regression analysis. This statistical technique removes whatever factors the control variables and the marshmallow test have in common. These controls included measures of the child’s socioeconomic status, intelligence, personality, and behavior problems. As more and more factors were controlled for, the association between marshmallow waiting and academic achievement as a teenager became nonsignificant. 

c) What's proposed above is that social class is a third variable ("C") that might be associated with both waiting time ("A") and school achievement ("B"). Using Figure 8.15. draw this proposal. Think about it, too: Why does it make sense that lower SES might go both with lower waiting time (A)? Why might lower SES go with lower school achievement (B)? 

d) Now create a mockup regression table that might fit the pattern of results being described above. Put the DV at the top (what is the DV?), then list the predictor variables underneath, starting with Waiting time at Age 4, and including things like Child's Socioeconomic Status and Intelligence. Which betas should be significant? Which should not?

Basically, here we have a core bivariate relationship (between wait time and later achievement), and then a critic suggests a possible third variable (SES). They used regression to see if the core relationship was still there when the third variable was controlled for. The core relationship went away, suggesting that SES was a third variable that can help explain why kids who wait longer do better in school later on. 

Next let's talk about some of the hype around this replication study. The Behavioral Scientist piece (quoted above) is one of the more balanced descriptions. Its headline was, Try to Resist Misinterpreting the Marshmallow Test. It emphasized that the core relationship was replicated. It also explains in some detail why SES is related to self-control, and how the two probably cannot be meaningfully separated--it's a nuanced report. But other press coverage had a doomsday feel:

Vox's subtitle read:
The famous psychology test gets roasted in the new era of replication.

The Guardian blared:
Famed impulse control 'marshmallow test' fails in new research

And the Quartz website said:
We learned the wrong lesson about self-control from the famous marshmallow test

One person on Twitter even wrote,
"The marshmallow/delayed gratification study always felt "wrong" to me - this year it was reported to be hopelessly flawed"

Are these headlines and comments fair? Probably not. As Payne and Sheeran write in Behavioral Scientist, 

The problem is that scholars have known for decades that affluence and poverty shape the ability to delay gratification. Writing in 1974, Mischel observed that waiting for the larger reward was not only a trait of the individual but also depended on people’s expectancies and experience. If researchers were unreliable in their promise to return with two marshmallows, anyone would soon learn to seize the moment and eat the treat. He illustrated this with an example of lower-class black residents in Trinidad who fared poorly on the test when it was administered by white people, who had a history of breaking their promises. Following this logic, multiple studies over the years have confirmed that people living in poverty or who experience chaotic futures tend to prefer the sure thing now over waiting for a larger reward that might never come. But if this has been known for years, where is the replication crisis?

e) Give at least 2 reasons why online sources might use incorrect headlines when they cover psychological science. 

f) Check out at least two different press stories about the marshmallow replication effect, and see how similar they are to each other and to the summary you analyzed above, by Payne and Sheeran. 

You can read the published paper for yourself! The full replication paper is in Psychological Science and is open access. 

Engagement with Disney Princesses

The study's results showed that the more children were engaged with Disney Princess media, the more they adhered to stereotypically female play later on. Photo: Shutterstock

There's a lot to think about in a recent study on Disney Princess Media. The study was covered by the Huffington Post here, and by another source in Oregon.  (You'll find other press coverage if you search for it).  The study illustrates some of the concepts in Chapter 8 and 9.

Let's point out the causal claims made by the journalists first. The Huffington Post journalist's headline reads:

Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys), Study Finds

And the Oregon paper's headline reads:

Disney 'princess culture' damaging to young girls, study says

Did you notice the causal verbs, "impact" and "damaging"? That's probably inappropriate, because this was a correlational study. In the first paragraph, the HuffPo journalist continues the dramatic causal language: 

Brigham Young University family life professor Sarah M. Coyne conducted a study with 198 preschoolers, both male and female, to rank their interaction with Disney Princess culture — including toys and movies — and how that affected their behavior one year later with reports from parents and teachers. 

a) Look back at that quote and find the verb that makes the causal claim. 

 

Now let's talk about the moderator in the headline, Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys).

b) This headline describes a moderator. Review what you learned about moderators in Chapter 8. What is the core relationship in this example (what are the two core variables)? And what is the moderator variable? How would you sketch this headline in a table or figure? 

 

Now, here are some more points made about the study, quoted from the Oregon news source:

Linder and her co-authors found that 96 percent of girls and 87 percent of boys had taken in some form of princess culture. Roughly 61 percent of girls reported playing with a princess doll in the last week, but only 4 percent of boys said they had played with the dolls.

When the subjects were examined a year after the initial analysis, both girls and boys who played with the dolls showed an increased likelihood to adhere to female gender stereotypes, a fact that could prove especially problematic for girls, said Sara Coyne, a co-author of the study and professor of family life at Brigham Young University.

Here's how the HuffPo journalist described this part of the results:

For both boys and girls, greater interactions with princesses predicted more gender-normative behavior — like wanting to play with traditional female stereotypical toys and activities (e.g., dolls, tea sets, playing house), and behaviors (e.g., avoiding getting dirty, avoiding taking risks)

c) What are the variables in this study, as described above? Are they measured or manipulated? And given your answer, does that make the study correlational or experimental? Did you notice that the design is longitudinal? (What signaled you to that?) Which design in Chapter 9 does this design seem closest to? 

d) Now, do these descriptions of the results actually show a relationship between Disney Princess media and gender-normative behavior that is different for boys and girls, as the one headline suggested? 

 

OK. If the relationship between Disney culture and stereotypical behavior was actually the same for boys and girls, where did the moderator in the headline come from? Turns out that when the researcher was interviewed about the results, she indicated that the gender stereotypes could, in the future, lead to different outcomes for boys and girls. Therefore, the moderator in the headline came more from the researcher's discussion than from the study itself. The researcher said, for example:

"We know that girls who strongly adhere to female gender stereotypes feel like they can't do some things," she told the university's news site. "They're not as confident that they can do well in math and science. They don't like getting dirty, so they're less likely to try and experiment with things."

And for boys:

The effects of princess culture were less troubling for boys, the authors said, as more stereotypically feminine behavior could serve as a counterbalance to the hyper-masculine superhero culture many young boys take in on a daily basis.

 

If your college or university subscribes to the journal, Child Development,you can see the full report of the study and learn more about the full pattern of results!

 

 

Social lives of the rich

The wealthier people were, the more likely they were to spend time alone. Photo: Shutterstock

In some earlier entries on this blog, I've posted stories about how money is related to (or can even change) people's social behavior, such as here and here. This week, the Washington Post ran a story about the association between money and socialization patterns. Let'sl= review the study, and you can have the chance to decide if the journalist made the right kinds of claims about the research.

The journalist first described how researchers Emily Bianchi of Emory University and Kathleen Vohs of the University of Minnesota tested the relationship between making money and socializing. People who make more money, they found, spend less time socializing with others:

 Bianchi and Vohs...looked at nearly 30,000 responses to the General Social Survey (GSS) to trace how household income affects respondents' social interactions. The GSS has repeatedly asked Americans how often they spend a "social evening" with relatives, neighbors and friends. Those questions draw a fairly robust portrait of Americans' social interactions. Bianchi and Vohs controlled for age, race, gender, marital status, household size, city size and hours worked to isolate the effect of money on relationships.

a) What are the two key variables in the basic association?

b) The language used above, "controlled for" should tell you that they also analyzed the data using multiple regression. First, why was it important to control for variables such as "hours worked"?

c) Sketch out what the regression table might have looked like in their study. What would the dependent, or criterion variable, have been? What would the predictor variables have been? What will the beta be for "income"? Will it be positive or negative? Which other betas do you think will be significant, and what will their direction be? (You can make up some betas for your table to match your predictions.)

Next, here's an indicator of the effect size of the relationship:

Compared to a low-income individual (earning $5,000 a year), a person from a higher-income household ($131,000 a year) spends, on average, 6.4 fewer evenings each year socializing with other people — even after controlling for differences in hours worked.

d) What do you think of the practically-stated effect size of "6.4 fewer evenings per year"? Does that seem small or large, in a practical sense?  

Now, here are some of the sentences included in the story that were used to describe the results. For each one, decide if it is a causal statement or an association statement (remember to attend to the verbs!):

e)  "How money changes our social networks"

f) "Inflation-adjusted household income has risen considerably since the World War II era (even if it's been fairly stagnant in the past decade). And if the relationship between income and social behavior they describe is true, then it's only natural that decreasing social interactions will be one consequence of that change."

g) "as people make more money, they spend less time socializing with others"

As you have learned, a correlational study like this one, even though it controlled for several possible third variables, cannot support a causal claim. For one, there may not be temporal precedence because in surveys like the General Social Survey, people usually answer all the questions at the same time. In addition, there are always uncontrolled third variables that the study did not measure or include in the regression. What possible third variables can you think of that might be associated with both having money and socializing less?

 

Happiness and longevity: What about mediators?

One hypothesis is that healthy habits mediate the relationship between happiness and longevity. Could you draw that mediator argument? Photo: Canon Boy/Shutterstock

When I teach multivariate correlational designs, students always end up asking, "What's the difference between multiple regression and mediation?" 

Here's the better question. What they should be asking is,  "What is the difference between using multiple regression analyses to rule out third variables, and using multiple regression analyses to test for mediation?"   This LA times editorial  provides an excellent example to illustrate the difference.

First things first:  Lots of people believe that happy people live longer, and happiness researchers have previously found that happiness does predict longevity.  However, a large-scale study published in the Lancet recently concluded that, in fact, happy people do not live longer. Their study was conducted on nearly a million women in the UK, so it's nicknamed "the million women study." The press covered the study by saying, "Good news for the grumpy."  You can see one story about the big Lancet study here.

However, this LA Times editorial, written by three happiness psychologists, explained a problem with the Lancet study. The Lancet study used regression to look at the relationship between happiness at age 59 and their life span, while holding constant a variety of third variables, such as income, being in a romantic relationship, being religious, smoking, and exercising. From one perspective, this makes sense--they wanted to look for some relatively "pure" relationship between happiness and longevity controlling for these factors, and they did not find one. That is, happiness did not predict longevity when these variables were controlled for.

a) Students: can you sketch a regression table of the pattern described above? Label the criterion variable, the predictor variables, and think about what betas might have been associated with each predictor variable.

Now for the problem: The three happiness psychologists writing the editorial argued that by controlling for such variables as relationship status, smoking, and exercise, the Lancet study obscured the important features of the relationship--the mediators. Here's how they put it.

We ... have concerns with the authors' choice to statistically control so many other variables. In our work, we hypothesize that the key ways positive feelings influence health are through changes in behaviors. For example, research has shown that positive individuals sleep better, smoke less and exercise more, all of which are known to predict longevity. Thus, when the authors control for all these variables, they are taking away a major path through which happiness is likely to influence longevity. This is like trying to determine whether a hurricane causes damage after controlling for its wind speed. Again, the question is changed, from “Will happy people live longer” to “Will happy people live longer if we take away their healthier behavior?”

b) The authors are stating a mediator argument in the above paragraph. Draw the mediator pattern they are describing, using Figures 9.11 or 9.13 as models. (Hint: Draw only one proposed mediator at a time.)

When the authors of the editorial write, "we hypothesize that the key ways positive feelings influence health are through changes in behaviors" they are talking about a mediator. The "through" part is a way of stating the mediator.

The authors of the editorial, therefore, end up describing how multiple regression can be used for at least two purposes. Regression is simply a statistical tool, and it can be used for both ruling out third variable problems and for testing mediation. The Lancet article used regression to control for what it saw as "pesky" third variables such as smoking and relationship status to see if happiness, on its own, predicted longevity. In contrast, the three happiness researchers have used regression to see if mediators, such as smoking and relationship status, explain why happiness predicts longevity.

You might recall in Chapter 9's discussion of mediation, that mediation tests are conducted in several steps. Step 4, in fact, uses multiple regression (the text states, for example, "if physical activity is the mediator of the relationship [between recess and behavior problems,] then the relationship between recess and behavior problems should drop when physical activity is controlled for").

So there you go. A great example of how a single data set can be viewed in two very different, but ultimately two very legitimate ways, depending on one's research goals and hypotheses.

By the way, the three happiness psychologists also criticized the construct validity of the Lancet study's happiness measure, saying this:

the happiness measure used in the Lancet study is weak. In well-crafted research, social scientists think carefully about question and answer design, the ordering of items and how many questions are too many before the participant gives up and stops answering or stops answering honestly...The Million Women Study asked just one happiness question: How often do you feel happy (from “rarely/never” to “most of the time”). A better approach would have been to ask the question in more ways, with a broader range of positive emotions for participants to choose from.

"Turn off the TV, save your brain?"

Photo: Shutterstock

Pacific Standard does a great job covering social science in the news. Here's a nice example of multiple regression analyses,  in a study they headlined, "Turn off the TV, Save your brain?"

The basic story comes from a large study that found  the amount of TV people watched in their young-adult years was linked to lower cognitive abilities in middle age.

a) So far, this is a simple bivariate correlation. Can you sketch a well-labelled scatterplot of this? Is the relationship positive or negative? 

b) This bivariate correlation apparently has temporal precedence, because the TV variable was measured in young adulthood, before the cognitive abilities (in middle age). Can we make a causal claim now? If not, what third variables might be responsible for this relationship?

Here are some methodological details about the study. You'll see that the story shows the telltale signs of controlling for some of these possible third variables:

The study featured 3,247 people who were recruited from four American cities (Chicago, Minneapolis, Oakland, and Birmingham) in the late 1980s. All were between the ages of 18 and 30 at the study's outset. They were tracked for the next quarter century, with follow-up examinations every two to five years....once every five years, [the participants] indicated "the average number of hours per day (they) spent watching television in the past 12 months."Those who reported watching an average of more than three hours per day during more than two-thirds of their visits were categorized as having a "long-term pattern of high television viewing."

At year 25, the participants took three well-known cognitive tests: the Digit Symbol Substitution Test, which measures speed of mental processing; the Stroop Test, which measures processing speed and attention; and the Rey Auditory Verbal Learning Test, which assesses how well we remember information delivered orally.

Participants who showed a long-term pattern of high TV viewing (approximately 11 percent of the total) scored significantly lower, on average, on [the] cognitive tests...This held true after taking into consideration a range of factors, including age, race, education, alcohol use, and body mass index.

 

c) Based on the description above, you could sketch a multiple regression table depicting this study's results. What would be the criterion variable? What would be the predictor variables? What betas do you think would be significant? Which  betas will be positive? Which will be negative?

d) The description above also tells you how some of the variables were operationalized. For example, how did they operationalize television viewing? How did they operationalize cognitive ability?

e) Even though this study sure feels like it could support a causal claim, it can't. (Remember, no correlational study can definitively support a causal claim.)  That's because even though it has temporal precedence (TV was measured before cognitive ability), and controls for several third variables (including age, race, education, alcohol use, and BMI), they didn't control for all possible third variables. What are some additional third variables that were not controlled for in this study?

 

Correlates of economic disadvantage in girls and boys

Why might boys be more vulnerable to the psychological correlates of poverty? Photo: Albina Tiplyashina Shutterstock

Some psychologists and sociologists are interested in how children from different socioeconomic conditions develop psychologically. One recent press report in Yahoo started with this sobering but unsurprising summary of past research, namely:

For children, the effects of poverty can manifest in surprising ways. Recent research shows that growing up in stressful economic conditions can disrupt brain development, alter behavior, and challenge emotions. 

Ignoring the causal language for now, you could note that this paragraph indicates three possible target correlations: One between poverty and brain development, one between poverty and behavior, and one between poverty and emotions.

But then, the story notes the following:

But for boys, on the whole, the outcome is worse.

This indicates that gender is a moderator of the relationship between poverty and psychological outcomes. Let's go back to the story to find out more.

Researchers studied the health, academic, and disciplinary records of more than 1 million kids born in low-income households in Florida between 1992 and 2002. They found that boys born to single mothers with little education, living in low-income neighborhoods, and enrolled in poor-quality public schools experienced higher rates of truancy and behavioral issues into middle school compared with their sisters.

a) Based on the paragraph above, make a table of results similar to the moderator examples in Chapter 8, showing how this moderator might have been represented. You'll have to make up some correlations, but make sure they match the results indicated. Hint: Your moderator is gender, and your target relationship is the correlation between poverty and truancy/behavioral issues.

The story goes on in the next paragraph:

[Poorer] boys also showed higher rates of cognitive disability, scored lower on standardized tests, and were less likely to graduate from high school relative to girls. They were also more likely to encounter the juvenile justice system.

b) Make another moderator table (using the one you did in question a as a model) depicting one of the results in the above paragraph.

Why does gender moderate the relationship between poverty and outcomes? Importantly, the answer is NOT because "boys are more likely to be poor" or that "boys are more likely to be truant."   Instead, a moderator is about how the relationship between poverty and truancy is stronger for one group (boys) than the other (girls). For boys, poverty has a stronger effect. In still other words, boys are more vulnerable to poverty.

The researchers cited in the Yahoo story present a at least one guess about why gender is a moderator.  Notice that they use the term "vulnerability," which is a good one to use when you're talking about a moderator.

One explanation for boys being particularly vulnerable in single-mother, low socioeconomic households is that they lacked a father figure in the home. Researchers pointed to existing studies showing that parents typically spend more time with the child of their own sex—mothers with their daughters and fathers with their sons.

Presumably, by spending more time with their daughters than with their sons, moms living in poverty might be buffering the effects of poverty in girls, and leaving boys somewhat more vulnerable.

"Screentime drags down grades"

Photo: VGstockstudio/Shutterstock

Several news outlets picked up on a study about high school students' screentime and their grades. The Huffington Post headlined their story, "One Hour Of Extra Screen Time Drags Down Teenagers' Grades"

The summary of the study read like this:

Teenagers who spend an extra hour a day surfing the internet, watching TV or playing computer games risk performing two grades worse in exams than their peers who don't, according to research by British scientists.

An extra hour in front of the TV or online at age 14-and-a-half was linked with 9.3 fewer exam points at age 16 -- equivalent to two grades, for example from a B to a D. Two extra hours was linked to 18 fewer points.

a) Based only on this summary, what kind of study do you think this is--correlational or experimental? What are the two key variables in this study?

b) Since the study is correlational, you might be surprised that the headline was causal, saying that "screentime drags down grades." Apply the three criteria for causation to this claim, given what you know about the methodology. Does it support the claim?

Later in the article, the researcher explains the following about the study:

"We only measured [screen time at age 14.5], but this is likely to be a reliable snapshot of participants' usual behavior, so we can reasonably suggest that screen time may be damaging to a teenager's grades," said Kirsten Corder of Cambridge's Centre for Diet and Activity Research, who co-led the work.

c) In this quote, the researcher seems to be focusing on one of the three criteria for causation. Which one? Which one is she ignoring?

d) Challenge question: In the quote above, the researcher mentions "We only measured [screen time at age 14.5], but this is likely to be a reliable snapshot of participants' usual behavior".  What concept is the researcher discussing here? Is this concept relevant to the causal claim the researchers and journalists wish to make?

e) Does this study appear to be a cross-lag panel design? Why or why not?

Suggested answers:

a) This study is most likely correlational, because it seems very likely that the two key variables in the study--amount of time spent on screens and grades earned--were measured.

b) Covariance: Yes, the study's results do suggest that more screen time is associated with lower grades.

Temporal precedence: Yes, the study measured screen time two years before grades.

Internal validity: It's not clear if any third variables were controlled for in a regression analysis. Even if they were, a regression analysis cannot control for all possible third variables, so this study cannot achieve internal validity. For example, it's possible that boys spend more time on screens and also get lower grades, so gender could be a third variable. Students with less parental support might both spend more time on screens and achieve lower grades, so parental support could be a third variable, too.

c) By emphasizing that screen time was measured at age 14.5, the authors seem to be emphasizing the study's temporal precedence, but not its internal validity.

d) The comment is referring to the test-retest reliability of the screen time measure. The researcher is saying that she assumes that screen time will be a stable trait, and that it would have good test-retest reliability. Of course, test-retest reliability is an empirical question--you have to collect data to find out if it exists for some measure. Assuming is not enough. In addition, the test-retest reliability of a measure is a consideration for construct validity.  Having good construct validity is important for a study, but construct validity does not, by itself, help a study support a causal claim.

e) This is probably not a cross-lag panel design. In such a design, you need to measure the two key variables (screen time and grades) twice: once at Time 1 and again at Time 2. By saying that they only measured screen time once, the researcher is also revealing that this study is not a cross-lag panel design.