Everyday Research Methods

Really? Prenatal cannabis exposure and early language development

Photo: Viacheslav Lopatin/Shutterstock

A story in PsyPost was headlined with, Prenatal cannabinoid exposure appears to have a strange impact on early language development. 

You might have noticed that the verb phrase, "has a strange impact on" is a causal one. We are being asked to believe that prenatal exposure to cannabis causes something in children's early language development.  I hope your causality suspicions are activated here, since it's almost certain that both cannabis exposure and language development are measured variables. This was probably a correlational study. 

Let's read on to find out what the mysterious "strange impact" is.

The journalist reported on a study in the open-source journal, Frontiers in Pediatrics. The PsyPost journalist drops the "strange impact" in the first paragraph:

...infants exposed to cannabis in the womb tend to exhibit improved language development by their first birthday. This surprising finding sheds light on the impacts of prenatal cannabis exposure on one aspect of early childhood neurodevelopment. However, experts advise that women avoid cannabis use during pregnancy due to known risks and unknown long-term effects.

Given past research, it's a bit surprising to find a positive relationship between cannabis use and child development, so that was the "strange impact" PsyPost mentions in their headline.

Here's how the journalist summarized the study's methods:

The researchers recruited 207 pregnant individuals and their 12-month-old infants who were part of The Safe Passage Study. This was a large prospective study that followed participants from 2007 to 2015.

Talavera-Barber and colleagues assessed the infants according to the Mullen Scale of Early Learning, a tool that assesses cognitive development in children from 2 days old to 68 months. 

[...]Prenatal cannabis exposure was collected from the mothers through self-report. Participants were categorized based on whether the exposure to cannabis occurred early (first trimester only; 51 participants) or late (second or third trimester; 18 participants), and they were randomly matched with unexposed participants for comparison (138 participants).

Don't be misled by the term "randomly matched with unexposed participants." A peek at the full empirical article shows that these were simply people in the sample who did not use cannabis. They were randomly selected from a larger set of participants in the study--but that doesn't mean they were "randomly matched." (In fact, I'm not sure I know what the journalist meant by "randomly matched" here.)

Questions

a) Based on what you've read so far above, complete this table of the variables in the study:

Variable name (stated at the conceptual level)	How was this variable operationalized?	Is this a measured or a manipulated variable?

Some of the results were summarized this way by PsyPost:

Unexpectedly, infants who were exposed to cannabis later in pregnancy scored higher in both expressive and receptive language areas than those who were not exposed.

Additionally, infants exposed early in pregnancy showed better gross motor skills, although there was no difference in fine motor skills and visual reception skills.

b) Sketch a bar graph of the relationship between exposure to cannabis later in pregnancy and expressive language scores.

c) Now apply the three criteria for causation. Sketch three diagrams following the model in Chapter 3, assigning one variable (cannabis exposure) to A and another (expressive language) to B. 

d) For the internal validity step, think of a specific third variable, C, that goes both with high levels of cananbis exposure and with higher levels of language development. 

e) Notably, if you read the original empirical article, you can see that the researchers used multiple regression to control for some "C" variables (i.e., third variables). They controlled for age, race, education, household income, prenatal alcohol and tobacco use, parent's anxiety, and infant sex. That means if you used one of these variables as your "C" variable in question d, we can probably rule it out. Can you think of an additional "C" variable--one that they did not control for and that is plausibly associated with both cannabis use and higher language development? 

When questionable survey methods reach the Supreme Court

According to the report, the Court may have been misled by the quality and true findings of survey on the habits of gun owners. 
Photo: B Christopher/Alamy Stock Photo

Polling research can play a critical role in shaping public opinion and public policy, so it's crucial that it be conducted well. Polling (survey) research is increasingly used in court cases, too. Because survey research can have an important impact, it's important to scrutinize a survey's research methods.

An example from the United States is a landmark gun-rights case (known as the Bruen case) from 2022, one of many pro-gun rulings in recent years.  In the Bruen case, the Court ruled that New York State could not outlaw concealed-carry handguns. In their written opinions, several of the justices cited polling research that supposedly found that  guns are "commonly" used for self-defense. 

The full story can be found here, and it was covered in the podcast, The Daily, on June 20, 2024 (available here).

According to reporting by New York Times journalists, the U.S. Supreme Court's decision in the Bruen case depended, in part, on the results of expert research: a survey of gun owners conducted by Georgetown business professor William English. English's survey data was used to argue that many gun owners use guns for self-defense. The Court accepted the research as expert testimony and allowed it to influence their reasoning.  However, when journalists started looking into the survey and the researcher who conducted it, they found instances of questionable scientific practice.

First, the researcher who conducted the survey has not made himself available to journalists to answer questions about his research and he also did not respond to requests that he testify in court about his work on the survey. This reticence to answer questions about one's research goes against scientific norms of transparency, or as Merton called it, communality. As the textbook says, "Scientists should transparently and freely share the results of their work with other scientists and the public", p. 15. From the journalist:

Government lawyers tried to talk with [Dr.English] in December, after his research surfaced in a Firearms Policy Coalition lawsuit seeking to overturn Washington State’s ban on assault weapons. But Dr. English did not respond to emails, phone calls or a certified letter, and a process server bearing a subpoena could not find him at the university or his home, where someone inside “looked at the door, then turned away,” according to court records.

Faced with efforts to compel his testimony, the pro-gun plaintiffs in [a different] Washington case chose to drop all references to his research.

Second, the Dr. English misled people about his impartiality as a scientist. Specifially, the journalists learned that the surveys he conducted were funded by pro-gun legal groups connected to the National Rifle Association. Failing to disclose one's funding is a potential conflict of interest and violates Merton's norm of disinterestedness, which says "scientists strive to discover the truth, whatever it is; they are not swayed by conviction, idealism, politics, or profit," (p. 15). From the story:

Dr. English seems at first glance to be an impartial researcher interested in data-driven insights. He has said his “scholarly arc” focuses on good public policy, and his lack of apparent ties to the gun lobby has lent credibility to his work.

But Dr. English’s interest in firearms is more than academic: He has received tens of thousands of dollars as a paid expert for gun rights advocates, and his survey work, which he says was part of a book project, originated as research for a National Rifle Association-backed lawsuit....

At least one of Dr. English's papers on the gun survey in question (which is published here) does not disclose this funding. 

Third, the journalists reported that Dr English's research methods probably led to misleading results and interpretations of the data. Specifically, the journalists reported:

... the wording of some questions could elicit answers overstating defensive gun use, and that he cherry-picked pro-gun responses.

An example comes from the self-defense question on the survey. In reporting his results, English asserted that 31% of his survey respondents said "yes" when asked, "Have you ever defended yourself or your property with a firearm, even if it was not fired or displayed?" But in that presentation, English left out the full text of the question. The Times report highlighted in blue the omitted portions:

Many policymakers recognize that a large number of people participate in shooting sports but question how often guns are used for self-defense. Have you ever defended yourself or your property with a firearm, even if it was not fired or displayed? Please do not include military service, police work, or work as a security guard.

In Chapter 6, you can read that this is a potential example of a leading question. Gun owners might be more motivated to answer "yes" to it, given its lead-in.

Here's another example of leading question wording (with a leading portion highlighted in blue), coupled with English's misleading presentation of results:

Some have argued that few gun owners actually want or use guns that are commonly classified as ‘assault weapons.’ Have you ever owned an AR-15 or similarly styled rifle? You can include any rifles of this style that have been modified or moved to be compliant with local law. Answering this will help us establish how popular these types of firearms are.

Elsewhere in his findings, Dr. English quotes about 30 gun owners attesting to the usefulness of large-capacity magazines for self-defense — but only two reported actually firing a gun, both at animals. He did not quote any of the hundreds of negative responses from gun owners, including: “There is no reason to have a magazine like that unless you’re fighting in a war.”

 

The journalist concludes that while English's research was only one piece of the full argument in Bruen and other pro-gun rulings, "...his credentials and findings carried weight. At least five briefs cited his research."

Reflection Questions:

a) What's the difference between writing a leading survey question and writing a misleading summary of one's research? 

b) What do you think might have happened in the court cases if Dr. English had been more up-front about who funded his research? 

c) What do you think might have happened in the court cases if briefs about Dr. English's presentation of his research had included the full texts of the questions he used?