Everyday Research Methods

Does smoking marijuana cause car fatalities?

The study found an estimated 12% higher rate of fatal accidents after 4:20pm on April 20. Credit: Lars Hagberg/AFP/Getty Images

REPLICATION UPDATE (posted Feb 25, 2019) Another set of researchers wondered why the study below compared April 20 to April 13 and April 27, rather than comparing April 20 to every other day in April (or to every other day in the year). As it turns out, when you compare April 20 to every other day of the month or year (not just the two originally selected) April 20 does not actually stand out in terms of fatal crash risk. 

This new analysis recategorizes this post. It used to be tagged an example of a quasi-experiment. Now it is a good example of (possible) cherry-picking, and an example of replication efforts. The newer published study's abstract includes these details:

Between 1992 and 2016, ‘4/20’ was associated with an increase in the number of drivers involved in fatal crashes (IRR 1.12, 95% CI 0.97 to 1.28) relative to control days 1 week before and after, but not when compared with control days 1 and 2 weeks before and after (IRR 1.05, 95% CI 0.92 to 1.28) or all other days of the year (IRR 0.98, 95% CI 0.88 to 1.10). Across all years we found little evidence to distinguish excess drivers involved in fatal crashes on 4/20 from routine daily variations.

There is little evidence to suggest population-wide effects of the annual cannabis holiday on the number of drivers involved in fatal traffic crashes.

 

Original post:

Here's a study that took advantage of "4-20", an unofficial holiday which people celebrate by holding pot-smoking parties starting at 4:20pm. Here's how the quasi-experiment was described in a New York Times story: 

Researchers used 25 years of data on car crashes in the United States in which at least one person died. They compared the number of fatal accidents between 4:20 p.m. and midnight on April 20 each year with accidents during the same hours one week before and one week after that date.

a) What are the "independent" and dependent variables in this study? (And why did I put independent variable in quotes?)

Here's how the journalist described the results:

Before 4:20 p.m. there was no difference between the number of fatalities on April 20 and the number on the nearby dates. But from 4:20 p.m. to midnight, there was a 12 percent increased risk of a fatal car crash on April 20 compared with the control dates. 

b)  Of the four quasi experimental designs, which seems to be the best fit: Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design? 

c) Sketch a graph of the results described. 

d) The Times reported that "The increased risk was particularly large in drivers 20 and younger." Why might the researchers have included this detail?

e) The Times's headline read, "Marijuana Use Tied to Fatal Car Crashes". What kind of claim is this? (Frequency, Association, or Cause?)

f) To what extent can these results support a causal claim about marijuana causing crashes? Apply the three causal criteria to this design and results. 

 

 

 

Does legal marijuana lower opioid prescriptions? A quasi-experiment

Legalizing marijuana is associated with lower rates of opioid prescriptions in those U.S. states. Photo: Gina Kelly/Alamy Stock Photo

Opioid addition is a major health crisis in the United States.  Deaths from overdose increased dramatically in the last 5 years. Opioid addiction sometimes starts when a person in pain is prescribed legal opioid drugs by a physician. Opioid prescriptions can also be sold illegally. For these reasons, opioid prescription rates are an indicator of opioid abuse in a particular region.  

Some public health researchers have investigated whether legalizing marijuana can reduce rates of opioid use and abuse. Marijuana is an alternative for controlling chronic pain that, according to many experts, has a lower addiction risk. Recently, researchers published two studies, both with quasi-experimental designs, that tested whether legalized marijuana could lower the rates of opioid prescriptions. Like many quasi-experiments, the researchers took advantage of a real-world situation: Some U.S. states have legalized marijuana and other states have not.

ABC news covered the the research. There were two studies with similar designs, but we'll focus on the first one:

One looked at trends in opioid prescribing under Medicaid, which covers low-income adults, between 2011 and 2016. It compared the states where [medical] marijuana laws took effect versus states without such laws.... 

Results showed that laws that let people use marijuana to treat specific medical conditions were associated with about a 6 percent lower rate [over the years studied] of opioid prescribing for pain. That's about 39 fewer prescriptions per 1,000 people using Medicaid.

And when states with such a law went on to also allow recreational marijuana use by adults, there was an additional drop averaging about 6 percent.

Questions:

a) What is the "independent" variable in this quasi-experiment? What is the dependent variable? Was the independent variable independent groups or within groups? 

b) What makes this a quasi-independent variable? 

c) Of the four quasi experimental designs, which seems to be the best fit: Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design?

d) How might you graph the results described above? 

e) To what extent can these data support the causal claim that "legalizing marijuana, either for medical use or recreational use, can lower the rates of opioid prescriptions in the Medicaid system"?

 

The two studies are available open-access through JAMA Internal Medicine. Here's the study using Medicare Part D prescription rates, and here's the study using Medicaid prescription rates. 

 

Suggested answers

a) The independent variable was whether a state had legalized marijuana or not. It was independent groups (states either had, or had not, legalized the drug). The dependent variable was the number of opioid prescription rates through Medicaid. Another variable, somewhat difficult to discern from the journalist's description, was year of study (from 2011 to 2016)

b) This IV was not manipulated/controlled by the experimenter. The researcher did not decide which states could legalize marijuana or not.

c) This is probably best characterized as a non-equivalent control group, pretest-posttest design. There were two types of states (legalized and not) and one main outcome variable: opioid prescriptions. The prescription rate was compared over time (from 2011 to 2016), making it pretest-posttest.

d) Your y-axis should have "opioid prescriptions" and the x-axis should include the years 2011 to 2016.  You could then have "States with legalization" and "States without legalization" as two different colored lines. 

e) The results of the study show covariance (States with legalized marijuana had lower opioid prescriptions). The fact that they compared opioid prescriptions over time (2011 to 2016) suggest that the design is able to establish temporal precedence. Presumably (although this is not clear from the articles), 2011 represents a year before many of the marijuana laws took effect and 2016 data occurred after the laws had been active.
As for internal validity, it's possible that states that legalize are different in systematic ways than states that do not. For example, states that legalize marijuana are more likely to be in the North and West, have lower poverty rates, and so on. However, the pretest-posttest design, in which they studied the "drop in opioid prescriptions over time" rather than "overall rate of opioid prescriptions" helps minimize some of these concerns. As with most quasi-experiments, causation is not a slam-dunk, because the experimenter does not have full control over the independent variable. 

"Talk deeply, be happy?" It seems to replicate...

Figure 8.1: Data from Mehl et al., (2010)

In Chapter 8, one of the examples features a study that found that the more "deep talk" people engage in (as measured by the EAR), the happier they reported being (Mehl et al., 2010) (see Figure 8.1). 

The same 2010 study also reported that the amount of engagement in small talk was associated with lower well being (this result is presented in Figure 8.9). 

Now a team of researchers (including many of the same researchers) have published a second study with similar methodology (Milek et al., in press). The team collected new data in a larger, more heterogeneous sample of U.S. adults. (the original study was only on college students.) The authors used Bayesian analytic techniques, including pooling the new samples with the sample from the 2010 study. You can view a preprint of the report here. It's in press at Psychological Science.

The new paper confirmed evidence for the "deep talk" effect. That is, substantive conversations were linked with greater well being, with a moderate effect size. But the team did not find evidence for the complementary effect of small talk. That is, in the new analysis, the estimate of the small talk was not different from zero. 

If you teach this example, it's worth updating students: The "deep talk" result in Figure 8.1 has been replicated, but one of the effects in in Figure 8.9 (the small talk effect) has not. 

Figure 8.9: Data from Mehl, et al., (2010)