Everyday Research Methods

Do kids do better in private schools?

What evidence would it take to convince us that it's the schools, not other factors, that are responsible for the outcomes of private school students? Photo: Image Source / Alamy Stock Photo

A large study has compared the outcomes of children who've attended private schools to those who've attended public schools.  A journalist summarized the report in the Washington Post. The study provides a nice example of how multivariate regression can be used to test third variable hypotheses. 

When we look simply at the educational acheivement of students in private schools vs. public schools, private school students have higher achievement scores. However, all such studies are correlational, because the two variables--Type of School and Level of Achievement--are measured. 

Therefore, such studies how covariance, because the results depict a relationship. The study may even show temporal precedence, because attending school presumably precedes the measure of achievement. However, such studies are weak on internal validity. We can think of several alternative explanations for why children in private schools are scoring higher. 

One major alternative explanation is socioeconomic status. Children from wealthier families are more likely to afford private schools. And in general, children from wealthier families tend to score higher on achievement tests. 

The Washington Post journalist quoted one of study's authors, Robert Pianta, who summed up the study's results this way:

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Questions

a) Draw little diagrams similar to those in Figure 8.15 (in the 3rd ed.) to depict the arguments being made in this study. What would A be? What about B? In the quote from Pianta, above, what would the C variable(s) be? 

b) The researchers used type of school (private vs. public), which is a categorical variable. But in some analyses, the researchers also used "number of years in private school" as an alternative version of this variable. Is "number of years in private school" categorical, ordinal, interval, or ratio data? 

c) Sketch a mock-up regression table with the criterion variable  at the top and predictors below (Use Table 9.1 as a model). Which variable do you think the researchers selected as the criterion (dependent) variable in their analyses? Which variable(s) would have been the predictors? 

d) Now that you know what the results were, think about how the beta associated with "number of years in private school"  would change when parental SES is added and removed from the regression analyses. 

Original journal article seems to be open-access. Give it a try!

Suggested answers

a) A and B would be Type of School and Level of Acheivement. It doesn't really  matter which one is A and which one is called B. C would be Family Income and/or Parental Education.

b) Ratio data (zero is meaningful in this scale because you could attend zero years of private school)

c) The criterion variable would be Achievement, and the predictors would be Number of Years of Private School, Family Income, and Parental Education.

d)When the Number of Years of Private School is on the table (in the analysis) by itself, its beta is likely to be positve and significant (more years of private school goes with higher achievement). When Family Income and Parental Education are added to the table, the beta for Number of Years of Private School should drop to zero.  This pattern of results is consistent with the argument that Family Income and Parental Education are the alternative explanation for the original relationship .

Delay of gratification (marshmallow study replication)

Her school achievement later in life can be predicted from her ability to wait for a treat (or by her family's SES). Photo: Manley099/Getty Images

There's a new replication study about the famous "marshmallow study",  and it's all over the popular press. You've probably heard of the original research: Kids are asked to sit alone in a room with a single marshmallow (or some other treat they like, such as pretzels). If the child can wait for up to 15 minutes until the experimenter comes back, they receive two marshmallows. But if they eat the first one early, they don't. As part of the original study, kids were tracked over several years. One of the key findings was that the longer children were able to wait at age 4, the better they were doing in school as teenagers. Psychologists have often used this study as an illustration of how self-control is related to important life outcomes. 

The press coverage of this year's replication study illustrates at least two things. First, it's a nice example of multiple regression.  Second, it's an example of how different media outlets assign catchy--but sometimes erroneous--headlines on the same study.

First, let's talk about the multiple regression piece. Regression analyses often try to understand a core bivariate relationship more fully. In this case, the core relationship they start with is between the two variables, "length of time kids waited at age 4" and "test performance at age 15." Here's how it was described by Payne and Sheeran in the online magazine Behavioral Scientist:

The result? Kids who resisted temptation longer on the marshmallow test had higher achievement later in life. The correlation was in the same direction as in Mischel’s early study. It was statistically significant, like the original study. The correlation was somewhat smaller, and this smaller association is probably the more accurate estimate, because the sample size in the new study was larger than the original. Still, this finding says that observing a child for seven minutes with candy can tell you something remarkable about how well the child is likely to do in high school. 

a) Sketch a well-labelled scatterplot of the relationship described above. What direction will the dots slope? Will they be fairly tight to a straight line, or spread out?

b) The writers (Payne and Sheeran) suggest that a larger sample size leads to a more accurate estimate of a correlation. Can you explain why a large sample size might give a more accurate statistical estimate? (Hint: Chapter 8 talks about outliers and sample size--see Figures 8.10 and 8.11.)

Now here's more about the study: 

The researchers next added a series of “control variables” using regression analysis. This statistical technique removes whatever factors the control variables and the marshmallow test have in common. These controls included measures of the child’s socioeconomic status, intelligence, personality, and behavior problems. As more and more factors were controlled for, the association between marshmallow waiting and academic achievement as a teenager became nonsignificant. 

c) What's proposed above is that social class is a third variable ("C") that might be associated with both waiting time ("A") and school achievement ("B"). Using Figure 8.15. draw this proposal. Think about it, too: Why does it make sense that lower SES might go both with lower waiting time (A)? Why might lower SES go with lower school achievement (B)? 

d) Now create a mockup regression table that might fit the pattern of results being described above. Put the DV at the top (what is the DV?), then list the predictor variables underneath, starting with Waiting time at Age 4, and including things like Child's Socioeconomic Status and Intelligence. Which betas should be significant? Which should not?

Basically, here we have a core bivariate relationship (between wait time and later achievement), and then a critic suggests a possible third variable (SES). They used regression to see if the core relationship was still there when the third variable was controlled for. The core relationship went away, suggesting that SES was a third variable that can help explain why kids who wait longer do better in school later on. 

Next let's talk about some of the hype around this replication study. The Behavioral Scientist piece (quoted above) is one of the more balanced descriptions. Its headline was, Try to Resist Misinterpreting the Marshmallow Test. It emphasized that the core relationship was replicated. It also explains in some detail why SES is related to self-control, and how the two probably cannot be meaningfully separated--it's a nuanced report. But other press coverage had a doomsday feel:

Vox's subtitle read:
The famous psychology test gets roasted in the new era of replication.

The Guardian blared:
Famed impulse control 'marshmallow test' fails in new research

And the Quartz website said:
We learned the wrong lesson about self-control from the famous marshmallow test

One person on Twitter even wrote,
"The marshmallow/delayed gratification study always felt "wrong" to me - this year it was reported to be hopelessly flawed"

Are these headlines and comments fair? Probably not. As Payne and Sheeran write in Behavioral Scientist, 

The problem is that scholars have known for decades that affluence and poverty shape the ability to delay gratification. Writing in 1974, Mischel observed that waiting for the larger reward was not only a trait of the individual but also depended on people’s expectancies and experience. If researchers were unreliable in their promise to return with two marshmallows, anyone would soon learn to seize the moment and eat the treat. He illustrated this with an example of lower-class black residents in Trinidad who fared poorly on the test when it was administered by white people, who had a history of breaking their promises. Following this logic, multiple studies over the years have confirmed that people living in poverty or who experience chaotic futures tend to prefer the sure thing now over waiting for a larger reward that might never come. But if this has been known for years, where is the replication crisis?

e) Give at least 2 reasons why online sources might use incorrect headlines when they cover psychological science. 

f) Check out at least two different press stories about the marshmallow replication effect, and see how similar they are to each other and to the summary you analyzed above, by Payne and Sheeran. 

You can read the published paper for yourself! The full replication paper is in Psychological Science and is open access. 

Engagement with Disney Princesses

The study's results showed that the more children were engaged with Disney Princess media, the more they adhered to stereotypically female play later on. Photo: Shutterstock

There's a lot to think about in a recent study on Disney Princess Media. The study was covered by the Huffington Post here, and by another source in Oregon.  (You'll find other press coverage if you search for it).  The study illustrates some of the concepts in Chapter 8 and 9.

Let's point out the causal claims made by the journalists first. The Huffington Post journalist's headline reads:

Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys), Study Finds

And the Oregon paper's headline reads:

Disney 'princess culture' damaging to young girls, study says

Did you notice the causal verbs, "impact" and "damaging"? That's probably inappropriate, because this was a correlational study. In the first paragraph, the HuffPo journalist continues the dramatic causal language: 

Brigham Young University family life professor Sarah M. Coyne conducted a study with 198 preschoolers, both male and female, to rank their interaction with Disney Princess culture — including toys and movies — and how that affected their behavior one year later with reports from parents and teachers. 

a) Look back at that quote and find the verb that makes the causal claim. 

 

Now let's talk about the moderator in the headline, Disney Princesses May Impact Gender Stereotypes For Girls (But Not Boys).

b) This headline describes a moderator. Review what you learned about moderators in Chapter 8. What is the core relationship in this example (what are the two core variables)? And what is the moderator variable? How would you sketch this headline in a table or figure? 

 

Now, here are some more points made about the study, quoted from the Oregon news source:

Linder and her co-authors found that 96 percent of girls and 87 percent of boys had taken in some form of princess culture. Roughly 61 percent of girls reported playing with a princess doll in the last week, but only 4 percent of boys said they had played with the dolls.

When the subjects were examined a year after the initial analysis, both girls and boys who played with the dolls showed an increased likelihood to adhere to female gender stereotypes, a fact that could prove especially problematic for girls, said Sara Coyne, a co-author of the study and professor of family life at Brigham Young University.

Here's how the HuffPo journalist described this part of the results:

For both boys and girls, greater interactions with princesses predicted more gender-normative behavior — like wanting to play with traditional female stereotypical toys and activities (e.g., dolls, tea sets, playing house), and behaviors (e.g., avoiding getting dirty, avoiding taking risks)

c) What are the variables in this study, as described above? Are they measured or manipulated? And given your answer, does that make the study correlational or experimental? Did you notice that the design is longitudinal? (What signaled you to that?) Which design in Chapter 9 does this design seem closest to? 

d) Now, do these descriptions of the results actually show a relationship between Disney Princess media and gender-normative behavior that is different for boys and girls, as the one headline suggested? 

 

OK. If the relationship between Disney culture and stereotypical behavior was actually the same for boys and girls, where did the moderator in the headline come from? Turns out that when the researcher was interviewed about the results, she indicated that the gender stereotypes could, in the future, lead to different outcomes for boys and girls. Therefore, the moderator in the headline came more from the researcher's discussion than from the study itself. The researcher said, for example:

"We know that girls who strongly adhere to female gender stereotypes feel like they can't do some things," she told the university's news site. "They're not as confident that they can do well in math and science. They don't like getting dirty, so they're less likely to try and experiment with things."

And for boys:

The effects of princess culture were less troubling for boys, the authors said, as more stereotypically feminine behavior could serve as a counterbalance to the hyper-masculine superhero culture many young boys take in on a daily basis.

 

If your college or university subscribes to the journal, Child Development,you can see the full report of the study and learn more about the full pattern of results!

 

 

Social lives of the rich

The wealthier people were, the more likely they were to spend time alone. Photo: Shutterstock

In some earlier entries on this blog, I've posted stories about how money is related to (or can even change) people's social behavior, such as here and here. This week, the Washington Post ran a story about the association between money and socialization patterns. Let'sl= review the study, and you can have the chance to decide if the journalist made the right kinds of claims about the research.

The journalist first described how researchers Emily Bianchi of Emory University and Kathleen Vohs of the University of Minnesota tested the relationship between making money and socializing. People who make more money, they found, spend less time socializing with others:

 Bianchi and Vohs...looked at nearly 30,000 responses to the General Social Survey (GSS) to trace how household income affects respondents' social interactions. The GSS has repeatedly asked Americans how often they spend a "social evening" with relatives, neighbors and friends. Those questions draw a fairly robust portrait of Americans' social interactions. Bianchi and Vohs controlled for age, race, gender, marital status, household size, city size and hours worked to isolate the effect of money on relationships.

a) What are the two key variables in the basic association?

b) The language used above, "controlled for" should tell you that they also analyzed the data using multiple regression. First, why was it important to control for variables such as "hours worked"?

c) Sketch out what the regression table might have looked like in their study. What would the dependent, or criterion variable, have been? What would the predictor variables have been? What will the beta be for "income"? Will it be positive or negative? Which other betas do you think will be significant, and what will their direction be? (You can make up some betas for your table to match your predictions.)

Next, here's an indicator of the effect size of the relationship:

Compared to a low-income individual (earning $5,000 a year), a person from a higher-income household ($131,000 a year) spends, on average, 6.4 fewer evenings each year socializing with other people — even after controlling for differences in hours worked.

d) What do you think of the practically-stated effect size of "6.4 fewer evenings per year"? Does that seem small or large, in a practical sense?  

Now, here are some of the sentences included in the story that were used to describe the results. For each one, decide if it is a causal statement or an association statement (remember to attend to the verbs!):

e)  "How money changes our social networks"

f) "Inflation-adjusted household income has risen considerably since the World War II era (even if it's been fairly stagnant in the past decade). And if the relationship between income and social behavior they describe is true, then it's only natural that decreasing social interactions will be one consequence of that change."

g) "as people make more money, they spend less time socializing with others"

As you have learned, a correlational study like this one, even though it controlled for several possible third variables, cannot support a causal claim. For one, there may not be temporal precedence because in surveys like the General Social Survey, people usually answer all the questions at the same time. In addition, there are always uncontrolled third variables that the study did not measure or include in the regression. What possible third variables can you think of that might be associated with both having money and socializing less?

 

Happiness and longevity: What about mediators?

One hypothesis is that healthy habits mediate the relationship between happiness and longevity. Could you draw that mediator argument? Photo: Canon Boy/Shutterstock

When I teach multivariate correlational designs, students always end up asking, "What's the difference between multiple regression and mediation?" 

Here's the better question. What they should be asking is,  "What is the difference between using multiple regression analyses to rule out third variables, and using multiple regression analyses to test for mediation?"   This LA times editorial  provides an excellent example to illustrate the difference.

First things first:  Lots of people believe that happy people live longer, and happiness researchers have previously found that happiness does predict longevity.  However, a large-scale study published in the Lancet recently concluded that, in fact, happy people do not live longer. Their study was conducted on nearly a million women in the UK, so it's nicknamed "the million women study." The press covered the study by saying, "Good news for the grumpy."  You can see one story about the big Lancet study here.

However, this LA Times editorial, written by three happiness psychologists, explained a problem with the Lancet study. The Lancet study used regression to look at the relationship between happiness at age 59 and their life span, while holding constant a variety of third variables, such as income, being in a romantic relationship, being religious, smoking, and exercising. From one perspective, this makes sense--they wanted to look for some relatively "pure" relationship between happiness and longevity controlling for these factors, and they did not find one. That is, happiness did not predict longevity when these variables were controlled for.

a) Students: can you sketch a regression table of the pattern described above? Label the criterion variable, the predictor variables, and think about what betas might have been associated with each predictor variable.

Now for the problem: The three happiness psychologists writing the editorial argued that by controlling for such variables as relationship status, smoking, and exercise, the Lancet study obscured the important features of the relationship--the mediators. Here's how they put it.

We ... have concerns with the authors' choice to statistically control so many other variables. In our work, we hypothesize that the key ways positive feelings influence health are through changes in behaviors. For example, research has shown that positive individuals sleep better, smoke less and exercise more, all of which are known to predict longevity. Thus, when the authors control for all these variables, they are taking away a major path through which happiness is likely to influence longevity. This is like trying to determine whether a hurricane causes damage after controlling for its wind speed. Again, the question is changed, from “Will happy people live longer” to “Will happy people live longer if we take away their healthier behavior?”

b) The authors are stating a mediator argument in the above paragraph. Draw the mediator pattern they are describing, using Figures 9.11 or 9.13 as models. (Hint: Draw only one proposed mediator at a time.)

When the authors of the editorial write, "we hypothesize that the key ways positive feelings influence health are through changes in behaviors" they are talking about a mediator. The "through" part is a way of stating the mediator.

The authors of the editorial, therefore, end up describing how multiple regression can be used for at least two purposes. Regression is simply a statistical tool, and it can be used for both ruling out third variable problems and for testing mediation. The Lancet article used regression to control for what it saw as "pesky" third variables such as smoking and relationship status to see if happiness, on its own, predicted longevity. In contrast, the three happiness researchers have used regression to see if mediators, such as smoking and relationship status, explain why happiness predicts longevity.

You might recall in Chapter 9's discussion of mediation, that mediation tests are conducted in several steps. Step 4, in fact, uses multiple regression (the text states, for example, "if physical activity is the mediator of the relationship [between recess and behavior problems,] then the relationship between recess and behavior problems should drop when physical activity is controlled for").

So there you go. A great example of how a single data set can be viewed in two very different, but ultimately two very legitimate ways, depending on one's research goals and hypotheses.

By the way, the three happiness psychologists also criticized the construct validity of the Lancet study's happiness measure, saying this:

the happiness measure used in the Lancet study is weak. In well-crafted research, social scientists think carefully about question and answer design, the ordering of items and how many questions are too many before the participant gives up and stops answering or stops answering honestly...The Million Women Study asked just one happiness question: How often do you feel happy (from “rarely/never” to “most of the time”). A better approach would have been to ask the question in more ways, with a broader range of positive emotions for participants to choose from.

"Turn off the TV, save your brain?"

Photo: Shutterstock

Pacific Standard does a great job covering social science in the news. Here's a nice example of multiple regression analyses,  in a study they headlined, "Turn off the TV, Save your brain?"

The basic story comes from a large study that found  the amount of TV people watched in their young-adult years was linked to lower cognitive abilities in middle age.

a) So far, this is a simple bivariate correlation. Can you sketch a well-labelled scatterplot of this? Is the relationship positive or negative? 

b) This bivariate correlation apparently has temporal precedence, because the TV variable was measured in young adulthood, before the cognitive abilities (in middle age). Can we make a causal claim now? If not, what third variables might be responsible for this relationship?

Here are some methodological details about the study. You'll see that the story shows the telltale signs of controlling for some of these possible third variables:

The study featured 3,247 people who were recruited from four American cities (Chicago, Minneapolis, Oakland, and Birmingham) in the late 1980s. All were between the ages of 18 and 30 at the study's outset. They were tracked for the next quarter century, with follow-up examinations every two to five years....once every five years, [the participants] indicated "the average number of hours per day (they) spent watching television in the past 12 months."Those who reported watching an average of more than three hours per day during more than two-thirds of their visits were categorized as having a "long-term pattern of high television viewing."

At year 25, the participants took three well-known cognitive tests: the Digit Symbol Substitution Test, which measures speed of mental processing; the Stroop Test, which measures processing speed and attention; and the Rey Auditory Verbal Learning Test, which assesses how well we remember information delivered orally.

Participants who showed a long-term pattern of high TV viewing (approximately 11 percent of the total) scored significantly lower, on average, on [the] cognitive tests...This held true after taking into consideration a range of factors, including age, race, education, alcohol use, and body mass index.

 

c) Based on the description above, you could sketch a multiple regression table depicting this study's results. What would be the criterion variable? What would be the predictor variables? What betas do you think would be significant? Which  betas will be positive? Which will be negative?

d) The description above also tells you how some of the variables were operationalized. For example, how did they operationalize television viewing? How did they operationalize cognitive ability?

e) Even though this study sure feels like it could support a causal claim, it can't. (Remember, no correlational study can definitively support a causal claim.)  That's because even though it has temporal precedence (TV was measured before cognitive ability), and controls for several third variables (including age, race, education, alcohol use, and BMI), they didn't control for all possible third variables. What are some additional third variables that were not controlled for in this study?

 

Correlates of economic disadvantage in girls and boys

Why might boys be more vulnerable to the psychological correlates of poverty? Photo: Albina Tiplyashina Shutterstock

Some psychologists and sociologists are interested in how children from different socioeconomic conditions develop psychologically. One recent press report in Yahoo started with this sobering but unsurprising summary of past research, namely:

For children, the effects of poverty can manifest in surprising ways. Recent research shows that growing up in stressful economic conditions can disrupt brain development, alter behavior, and challenge emotions. 

Ignoring the causal language for now, you could note that this paragraph indicates three possible target correlations: One between poverty and brain development, one between poverty and behavior, and one between poverty and emotions.

But then, the story notes the following:

But for boys, on the whole, the outcome is worse.

This indicates that gender is a moderator of the relationship between poverty and psychological outcomes. Let's go back to the story to find out more.

Researchers studied the health, academic, and disciplinary records of more than 1 million kids born in low-income households in Florida between 1992 and 2002. They found that boys born to single mothers with little education, living in low-income neighborhoods, and enrolled in poor-quality public schools experienced higher rates of truancy and behavioral issues into middle school compared with their sisters.

a) Based on the paragraph above, make a table of results similar to the moderator examples in Chapter 8, showing how this moderator might have been represented. You'll have to make up some correlations, but make sure they match the results indicated. Hint: Your moderator is gender, and your target relationship is the correlation between poverty and truancy/behavioral issues.

The story goes on in the next paragraph:

[Poorer] boys also showed higher rates of cognitive disability, scored lower on standardized tests, and were less likely to graduate from high school relative to girls. They were also more likely to encounter the juvenile justice system.

b) Make another moderator table (using the one you did in question a as a model) depicting one of the results in the above paragraph.

Why does gender moderate the relationship between poverty and outcomes? Importantly, the answer is NOT because "boys are more likely to be poor" or that "boys are more likely to be truant."   Instead, a moderator is about how the relationship between poverty and truancy is stronger for one group (boys) than the other (girls). For boys, poverty has a stronger effect. In still other words, boys are more vulnerable to poverty.

The researchers cited in the Yahoo story present a at least one guess about why gender is a moderator.  Notice that they use the term "vulnerability," which is a good one to use when you're talking about a moderator.

One explanation for boys being particularly vulnerable in single-mother, low socioeconomic households is that they lacked a father figure in the home. Researchers pointed to existing studies showing that parents typically spend more time with the child of their own sex—mothers with their daughters and fathers with their sons.

Presumably, by spending more time with their daughters than with their sons, moms living in poverty might be buffering the effects of poverty in girls, and leaving boys somewhat more vulnerable.

"Screentime drags down grades"

Photo: VGstockstudio/Shutterstock

Several news outlets picked up on a study about high school students' screentime and their grades. The Huffington Post headlined their story, "One Hour Of Extra Screen Time Drags Down Teenagers' Grades"

The summary of the study read like this:

Teenagers who spend an extra hour a day surfing the internet, watching TV or playing computer games risk performing two grades worse in exams than their peers who don't, according to research by British scientists.

An extra hour in front of the TV or online at age 14-and-a-half was linked with 9.3 fewer exam points at age 16 -- equivalent to two grades, for example from a B to a D. Two extra hours was linked to 18 fewer points.

a) Based only on this summary, what kind of study do you think this is--correlational or experimental? What are the two key variables in this study?

b) Since the study is correlational, you might be surprised that the headline was causal, saying that "screentime drags down grades." Apply the three criteria for causation to this claim, given what you know about the methodology. Does it support the claim?

Later in the article, the researcher explains the following about the study:

"We only measured [screen time at age 14.5], but this is likely to be a reliable snapshot of participants' usual behavior, so we can reasonably suggest that screen time may be damaging to a teenager's grades," said Kirsten Corder of Cambridge's Centre for Diet and Activity Research, who co-led the work.

c) In this quote, the researcher seems to be focusing on one of the three criteria for causation. Which one? Which one is she ignoring?

d) Challenge question: In the quote above, the researcher mentions "We only measured [screen time at age 14.5], but this is likely to be a reliable snapshot of participants' usual behavior".  What concept is the researcher discussing here? Is this concept relevant to the causal claim the researchers and journalists wish to make?

e) Does this study appear to be a cross-lag panel design? Why or why not?

Suggested answers:

a) This study is most likely correlational, because it seems very likely that the two key variables in the study--amount of time spent on screens and grades earned--were measured.

b) Covariance: Yes, the study's results do suggest that more screen time is associated with lower grades.

Temporal precedence: Yes, the study measured screen time two years before grades.

Internal validity: It's not clear if any third variables were controlled for in a regression analysis. Even if they were, a regression analysis cannot control for all possible third variables, so this study cannot achieve internal validity. For example, it's possible that boys spend more time on screens and also get lower grades, so gender could be a third variable. Students with less parental support might both spend more time on screens and achieve lower grades, so parental support could be a third variable, too.

c) By emphasizing that screen time was measured at age 14.5, the authors seem to be emphasizing the study's temporal precedence, but not its internal validity.

d) The comment is referring to the test-retest reliability of the screen time measure. The researcher is saying that she assumes that screen time will be a stable trait, and that it would have good test-retest reliability. Of course, test-retest reliability is an empirical question--you have to collect data to find out if it exists for some measure. Assuming is not enough. In addition, the test-retest reliability of a measure is a consideration for construct validity.  Having good construct validity is important for a study, but construct validity does not, by itself, help a study support a causal claim.

e) This is probably not a cross-lag panel design. In such a design, you need to measure the two key variables (screen time and grades) twice: once at Time 1 and again at Time 2. By saying that they only measured screen time once, the researcher is also revealing that this study is not a cross-lag panel design.

 

 

 

 

 

A mediator in the news! Political party affiliation, self-control, and free will

 

The researchers used a Stroop test like this one to operationalize self-control in their study. Photo: Poornima Venkataraman/Chttps://commons.wikimedia.org/wiki/File:Stroop_task.jpg

This story in the Pacific Standard covers an empirical investigation of political conservatism, self-control, and free will. Political conservatism is usually associated in the U.S. with the Republican Party, and is contrasted with liberalism (which is associated with the Democratic Party). Free will is the belief that humans are responsible for our own actions. Self-control is the ability to defer immediate impulses and temptations, often to focus on long-term goals.  Resisting cookies or ice cream requires self-control, as does resisting the urge to roll your eyes at a difficult person, or choosing to study rather than be distracted by a website or video game.

According to the story, correlational research has already established that conservatives endorse a belief in free will more than liberals do. And past research has also established that people who believe in free will have more self-control.  So the researchers tested all three of these concepts together in a single study:

...the study authors had participants complete a version of the Stroop test—a tricky task commonly used to evaluate psychological capabilities—in which the researchers presented the test takers with colored words on background colors that mismatched the words (for example, the word "blue" on a yellow background). The participants had to identify the color described by the word without getting distracted by the background color, and the speed with which they were able to do so gave researchers an idea of how well the participants could maintain focus on the task. The team also assessed the participants’ belief in free will and political ideology. As expected, the study found that, as conservatism increased, so too did belief in free will, and, consequently, measures of self-control improved.

a) What are the three variables in the study described above? For each variable, indicate the possible levels, and state whether each variable is measured or manipulated. 

b) The authors are quoted giving the following interpretation of their findings: he interpretation above fits the description of a mediator (Chapter 9). Sketch a diagram of this mediator, using arrows to connect the three variables (use Figure 9.11 as a model).

“[It] could be that conservatives believe they have greater control over their performance and thus expend greater self-control in their academic pursuits,” the study authors write.

 

Finally, the authors suggest an application of these findings, as follows:

These findings offer a new lens through which to view previous research that found differences between conservatives and liberals—say, the fact that conservative college students tend to get better grades than their liberal counterparts.

c) What other real-world behaviors might be related to self-control? Could you design a study to test correlational hypotheses about political beliefs and these self-control related behaviors?

 

Using Twitter posts to gauge the emotions of a community

Scientists used the words people used on twitter in different counties to measure the counties' emotional tone. © Christin Gilbert / AGE Fotostock

 

 

 

Researchers have used twitter posts to assess risks for cardiovascular disease in different communities. How did they do it?  Here's a summary of some of the methods, as summarized by Wray Herbert in the Huffington Post. As you read it, ask yourself, "what is the unit of analysis of this study?"  (Hint--the unit of analysis is not "the person," which it usually is in psychology. It's something different):

The Penn scientists are pioneers in an emerging field called digital epidemiology, and their aim is to use social media as a cheap and flexible method to assess the psychological traits -- and thus health risks -- of entire communities. To test this method's potential, the scientists collected 148 million tweets from across the U.S., sorted into their 1,347 counties of origin. The scientists also gathered socioeconomic and demographic data on these counties, which are home to 88 percent of Americans.

So the unit of analysis is not person, but county, right? That means that if you were to sketch a scatterplot from this study, each dot will represent a county.

Why were they interested in emotional content of the tweets from each county, anyway? Here's why:

Scientists have identified many of the key risk factors for heart disease, such as smoking, inactivity, obesity and hypertension, and these insights have significantly diminished risk of the world's leading killer. Psychological traits such as chronic stress and depression are also important risk factors, while optimism and social support are known to be protective. These psychological characteristics often affect entire communities, putting large numbers of people at risk for disease. 

Now, on to the results. You should be able to make a scatterplot of some of them--in fact, I'll ask you to do that in a minute.

They measured specific words and topics, both negative (hostility, cursing, aggression, boredom and fatigue) and positive (wonder, hope, triumph, opportunity), and used these linguistic patterns to characterize communities at risk for heart disease. They then compared these risk patterns to the actual mortality rates for each county, obtained from the Centers for Disease Control. [They found that}...negative emotions, disengagement and (especially) anger were all significantly correlated with heart disease. ... By contrast, positive emotions and engagement were associated with lower heart disease mortality. Engagement with life -- considered a key component of successful aging -- emerged as the most potent protective factor.

a) Sketch a well-labelled scatterplot of the correlation, described above, between negative emotions and heart disease.

b) Now sketch a well-labelled scatterplot of the correlation, described above, between anger and heart disease. (Compare it to the scatterplot you made in a)

c) Now sketch a well-labelled scatterplot of the correlation, described above, between positive emotions and heart disease mortality.

Here's an additional comment about the study.

This held true even after controlling for income and education, suggesting that Twitter language captures important information not accounted for by socioeconomic status.

d) The above quote should alert you that they used multiple regression in their analyses. Why would that be important for their investigation?

e) What would have been the DV, or criterion variable, in their regression analysis?  What would have been the predictor variables?

f) Challenge question!  The following quote from the story might tell you something important about the sizes of the betas in their regression table. What do you think it is telling you?

What's more, Twitter language was a better predictor of heart disease mortality than 10 common demographic and behavioral risk factors, including such infamous ones as smoking and high blood pressure.