Everyday Research Methods

Do kids do better in private schools?

What evidence would it take to convince us that it's the schools, not other factors, that are responsible for the outcomes of private school students? Photo: Image Source / Alamy Stock Photo

A large study has compared the outcomes of children who've attended private schools to those who've attended public schools.  A journalist summarized the report in the Washington Post. The study provides a nice example of how multivariate regression can be used to test third variable hypotheses. 

When we look simply at the educational acheivement of students in private schools vs. public schools, private school students have higher achievement scores. However, all such studies are correlational, because the two variables--Type of School and Level of Achievement--are measured. 

Therefore, such studies how covariance, because the results depict a relationship. The study may even show temporal precedence, because attending school presumably precedes the measure of achievement. However, such studies are weak on internal validity. We can think of several alternative explanations for why children in private schools are scoring higher. 

One major alternative explanation is socioeconomic status. Children from wealthier families are more likely to afford private schools. And in general, children from wealthier families tend to score higher on achievement tests. 

The Washington Post journalist quoted one of study's authors, Robert Pianta, who summed up the study's results this way:

“You only need to control for family income and there’s no advantage,” Pianta said in an interview. “So when you first look, without controlling for anything, the kids who go to private schools are far and away outperforming the public school kids. And as soon as you control for family income and parents’ education level, that difference is eliminated completely.”

Questions

a) Draw little diagrams similar to those in Figure 8.15 (in the 3rd ed.) to depict the arguments being made in this study. What would A be? What about B? In the quote from Pianta, above, what would the C variable(s) be? 

b) The researchers used type of school (private vs. public), which is a categorical variable. But in some analyses, the researchers also used "number of years in private school" as an alternative version of this variable. Is "number of years in private school" categorical, ordinal, interval, or ratio data? 

c) Sketch a mock-up regression table with the criterion variable  at the top and predictors below (Use Table 9.1 as a model). Which variable do you think the researchers selected as the criterion (dependent) variable in their analyses? Which variable(s) would have been the predictors? 

d) Now that you know what the results were, think about how the beta associated with "number of years in private school"  would change when parental SES is added and removed from the regression analyses. 

Original journal article seems to be open-access. Give it a try!

Suggested answers

a) A and B would be Type of School and Level of Acheivement. It doesn't really  matter which one is A and which one is called B. C would be Family Income and/or Parental Education.

b) Ratio data (zero is meaningful in this scale because you could attend zero years of private school)

c) The criterion variable would be Achievement, and the predictors would be Number of Years of Private School, Family Income, and Parental Education.

d)When the Number of Years of Private School is on the table (in the analysis) by itself, its beta is likely to be positve and significant (more years of private school goes with higher achievement). When Family Income and Parental Education are added to the table, the beta for Number of Years of Private School should drop to zero.  This pattern of results is consistent with the argument that Family Income and Parental Education are the alternative explanation for the original relationship .

Can greening up a vacant lot ease depression in a community?

The study randomly assigned lots like this one to be cleaned up, turned into green space, or left alone. Photo: 1000 Words/Shutterstock

When I first saw the headline, "Replacing Vacant Lots With Green Spaces Can Ease Depression In Urban Communities" I thought it was just another journalist putting a causal claim on a correlational study. So I was surprised to read this statement about the design:

[Researcher Eugenia South] and her colleagues wanted to see if the simple task of cleaning and greening these empty lots could have an impact on residents' mental health and well-being. So, they randomly selected 541 vacant lots [in the city of Philadelphia] and divided them into three groups.

They collaborated with the Pennsylvania Horticultural Society for the cleanup work.

The lots in one group were left untouched — this was the control group. The Pennsylvania Horticultural Society cleaned up the lots in a second group, removing the trash. And for a third group, they cleaned up the trash and existing vegetation, and planted new grass and trees. The researchers called this third set the "vacant lot greening" intervention.

Here's more:

The team surveyed residents living near the lots before and after their trial to assess their mental health and wellbeing. "We used a psychological distress scale that asked people how often they felt nervous, hopeless, depressed, restless, worthless and that everything was an effort," explains South.

The scale alone doesn't diagnose people with mental illness, but a score of 13 or higher suggests a higher prevalence of mental illness in the community, she says.

People living near the newly greened lots felt better. "We found a significant reduction in the amount of people who were feeling depressed," says South.

As one commentator noted:

Previous research has shown that green spaces are associated with better mental health, but this study is "innovative," says Rachel Morello-Frosch, a professor at the department of environmental science, policy and management at the University of California, Berkeley, who wasn't involved in the research.

"To my knowledge, this is the first intervention to test — like you would in a drug trial — by randomly allocating a treatment to see what you see," adds Morello-Frosch. 

Questions

a) How do we know that this is an experiment and not a correlational study?

b) What were the Independent and Dependent variables? 

c) What was the design: Posttest only? Pretest-posttest? Repeated measures? Or concurrent measures?

d) Sketch a graph of the results described above. 

e) Ask at least one question each about this study's construct, internal, external, and statistical validities.

f) Because this article was published in the open-access journal JAMA Network Open anyone can read the paper. Take a look carefully at the tables in the paper (especially Table 2). How strong do the results seem to you? Are the differences between the conditions large? Do you see improvements on all the measured variables, or just on a few of them? 

g) Why does the design of this study help support the causal claim that "Replacing vacant lots with green spaces can ease depression...."? (Apply the three causal criteria.)