Photo credit: blasbike/Deposit Photos
A journalist wrote about a recent meta-analysis in JAMA Psychiatry which evaluated the connection between weekly time spent exercising and levels of depression. The headline of the Washington Post summary read, "Exercising even half the recommended amount can help prevent depression". Let's delve into this research, as covered by the journalist in their brief report. You can access the original empirical work here.
Here's the opening message:
Already known to help ease depression, regular exercise may also help prevent it, with people who exercised just half the recommended weekly amount lowering their risk for depression by 18 percent.... However, those who were more active, meeting at least the minimum recommended physical activity level, reduced their risk for depression by 25 percent, compared with inactive people.
The empirical research was a meta analysis, and it seems that all of the studies involved longitudinal designs:
The findings stem from the analysis of data from 15 studies, involving 191,130 adults who were tracked for at least three years.
a) What are the steps you'd typically follow if you were conducting a meta-analysis?
According to the journalist, the recommended amount of weekly exercise is 150 minutes. Therefore, it seems that working out only 75 minute per week (by walking, for example) does predict lower rates of depression over time.
Let's apply the four big validities to this study.
b) Construct validity: We'd need to address the construct validity of each major variable--exercise and depression. It's likely you can measure exercise in multiple ways (self-report, observational, physiological). Can you tell from the journalist's report how the studies measured exercise?
What about depression--can you tell from the report how the studies measured depression?
I would assume that across the 15 studies, there were a variety of ways that depression and exercise were measured. Is this variability of operationalizations a strength of the meta-analysis, or a weakness?
c) Statistical validity. The journalist merely reports a single estimate: "people who exercised just half the recommended weekly amount lowering their risk for depression by 18 percent." But the empirical article gives us a 95% CI for this estimate: [13%, 23%]. In addition the journalist writes, "meeting at least the minimum recommended physical activity level, reduced their risk for depression by 25 percent, compared with inactive people." Again the empirical article gives the 95% CI as [18%, 32%%].
In your own words, explain what these two 95% CIs mean.
d) External validity. As we consider external validity, you might be tempted to focus on the total number of participants in the meta-analysis (more than 191,000 over 15 studies). Why would it be incorrect to use sample size to make decisions about external validity?
e) Internal validity. All 15 studies in this analysis were prospective, or longitudinal. What does that mean? Use the term "temporal precedence" in your response.
Now to focus on internal validity specifically: Can you think a possible third variable that might be associated with both exercise and depression over time? (It would be potentially important for the researchers to control for this variable if they hoped to support a causal claim.)
