Everyday Research Methods

Exercise and Depression: No effect after all?

A science blog reviewed a study that showed a null effect for exercise and depression. Although some past research--and certainly popular belief--have indicated that exercise can improve depressive symptoms, this study showed no effect:

Danish researchers Krogh and colleagues randomly 115 assigned depressed people to one of two exercise programs. One was a strenuous aerobic workout - cycling for 30 minutes, 3 times per week, for 3 months. The other was various stretching exercises.

The idea was that stretching was a kind of placebo control group on the grounds that, while it is an intervention, it's not the kind of exercise that gets you fit. It doesn't burn many calories, it doesn't improve your cardiovascular system, etc. Aerobic exercise is the kind that's most commonly been proposed as having an antidepressant effect.

So what happened? Not much. Both groups got less depressed but there was zero difference between the two conditions. The cyclists did get physically fitter than the stretchers, losing more weight and improving on other measures. But they didn't feel any better.

Chapter 10 addresses what we should think about when an experiment shows no significant effect. We can apply those categories to this example.

a) Specifically, does it seem that there was enough variability between levels?  Did the experimenter use a strong enough manipulation of exercise? What other questions might you ask here?

b) Or what if there was too much variability within each group? What questions might you ask to evaluate this possibility?

c) The blogger seems to conclude that perhaps exercise really doesn't affect depression. Do you think that is a reasonable conclusion too?

 

 

Suggested Answers

a) The study seems to have used a manipulation check, because they measured the physical fitness of the two groups after the period of exercise was over. The aerobic group was in better shape than the stretchers were, so it does seem that exercise was manipulated appropriately.

b) As the book explains, researchers use several techniques to reduce within-group variability. In this study, we might ask if the researchers used the most precise measure of depression that they could. We could also ask if the researchers attempted to keep situation noise to a minimum. If those steps were impossible, then another way to counteract within-groups variability is to use a within-groups design. This study used an independent-groups design, so researchers did not use this approach. 

When a within-groups design is not possible and a researcher is trying to control unsystematic variability due to situation noise and individual differences, the most common fix is to simply test more people. This study had about 65 people in each group. Although the number may seem small to you, a sample of this size is often enough to find a medium-sized effect of exercise. On the other hand, if the true effect of exercise on depression is very small, the researchers might have found a significant effect if they had used more than 100 people in each group.

(Appendix B in the book uses this analogy: If you're looking for an elephant in a dark room, you need only a small light--a small sample--to find it. If you're looking for an earring in that dark room, you'd need a large, powerful light--a large sample--to find it.)

c) If exercise had a medium strong effect on depression, it seems that this study would have found it. What do you think? What kind of further evidence would convince you that exercise does not help depression?

 

 

Data vs. intuition: which one best predicted the US election results?

The recent U.S. presidential election provided an interesting chance to compare the different  "ways of knowing" that are covered in Chapter 2.

That chapter explains why research is a better source for knowledge than intuition, authority, or personal experience.

Before the November 7, 2012 election, many strategists predicted the outcome. One such expert was Newt Gingrich, who predicted that Mitt Romney would win at least 300 electoral votes. He said,  "I believe the minimum result  will be 53-47 Romney, over 300 electoral votes, and the Republicans will pick up the Senate.  I base that on just years and years of experience." (Source)

In contrast, other strategists predicted the outcome based not on experience, but on statistical data (that is, on research!). The most famous expert in this camp was Nate Silver, whose blog, fivethirtyeight, uses statistical weighting to create an average of multiple opinion polls. Mr. Silver's prediction of the election was almost exactly right: He predicted--based only on research data from polls, not personal experience--that Obama would win 313 electoral votes. (Obama got 303, without counting Florida).

Silver even predicted that Florida would be the tightest state outcome--and it was. He predicted the vote there would favor Obama by less than a few hundredths of a percent, and the state is still undecided (but leaning Obama).

Earlier this week, Newt Gingrich apologized for his errors of prediction. Maybe next time, he will put aside his intuition and experience-driven judgments in favor of conclusions supported by scientific data.