Everyday Research Methods

Claim: "Drinking any amount of alcohol causes damage to the brain"

Photo credit: Dangubic/Deposit Photos

Here's a link to CNN's coverage of a study on alcohol use and gray matter in the brain. The headline reads, "Drinking any amount of alcohol causes damage to the brain, study finds." Let's consider how well the study supports the causal claim the journalist (or the editor) attached to it.  Here's one of CNN's introductory statements: 

In an observational study, which has not yet been peer-reviewed, researchers from the University of Oxford studied the relationship between the self-reported alcohol intake of some 25,000 people in the UK, and their brain scans.

a) The term, "observational study" in this context is synonymous with "correlational study." What are the key variables in the study? What does the quote above tell you about whether the variables were manipulated or measured? 

Here's the next paragraph in the CNN summary: 

The researchers noted that drinking had an effect on the brain's gray matter -- regions in the brain that make up "important bits where information is processed," according to lead author Anya Topiwala, a senior clinical researcher at Oxford.

"The more people drank, the less the volume of their gray matter," Topiwala said via email.

b) The CNN journalist erroneously wrote, "drinking had an effect on the brain's gray matter".  Let's walk through whether this correlational study here can support the journalist's causal claim. The first criterion is covariance, which is about the study's results. Does the study show covariance?

c) The second criterion is temporal precedence. The journalist doesn't provide information about whether the drinking behavior (variable A) was measured first in time, before brain scans measured people's gray matter (variable B). What are the two potential directions this relationship might go?

d) The third criterion is internal validity. There could be some third variable, C, which is associated with both A and B. See if you can come up with a potential C variable that could reasonably correlate with both amount of drinking and reduction in gray matter. 

You might have noticed that the opening to the CNN summary mentions that the study "has not yet been peer-reviewed". 

e) What happens during the peer review process? Do you think journalists should summarize, for the general public, studies that have not yet been peer-reviewed?

f) The empirical report has not yet been published, but you can read more about the study where it is posted on the preprint site, medarxiv. What are some pros and cons of putting one's research on "arxiv" sites like these? Consider issues such as openness of science, peer review, and public interest. 

 

Thanks again, Stephen Chew, for sharing the example!

Cat-in-the-box

Pancetta the cat sitting in Kanizsa square. Photo and research by Gabriella Smith, M.A.

I've shared several examples of research on pet dogs on this blog, so it's about time I present an example of research on pet cats.

If you own a cat yourself, you've probably noticed that they enjoy climbing into 3-dimensional boxes, baskets, and other containers. You might also know that cats also seem to prefer "climbing" into 2-dimensional boxes--that is, they are likely to sit in a simple square that has been masking-taped onto the floor. A recent study took a closer look at this adorable behavior as part of a test of whether cats are susceptible to a phenomenon known as the Kanisza contour illusion.

As illustrated in the photo of Pancetta that accompanies this post, the Kanisza contour illusion consists of four Pac-Man-like shapes that, when arranged just so, are usually interpreted by humans as a square. Researcher Gabriella Smith wondered if cats, too, would fall for the illusion, so she created an experiment to test it. 

An interesting aspect of Smith's study was that the cats participated in their own homes, an approach called "citizen science." In other words, the researchers  "let volunteers test their own cats at home in a standardized way and then send in the results".

Here's a description of the research study, as covered by NPR's All Things Considered.(The original empirical study was published in a journal published  by Elsevier, which means it's very difficult to obtain, even through universities.):

Participants received booklets with instructions on how to print out and cut up paper shapes that could be taped to the floor to create three options: the Kanizsa illusion, the actual outline of a square, and a control that had the Pac-Man shapes facing outward. The volunteers were instructed to videotape their cats as they encountered these stimuli in short tests over six days, while the volunteers wore sunglasses to avoid inadvertently giving the cats any signals with their eyes.

Their results? That the cats chose to sit in the illusory square as often as they did the actual outline of a square — and more than the non-square arrangement.

Here are some questions about the study.

a) This was an experiment. What was the independent variable (IV) and what were its levels?

b) What was the dependent variable (DV)?

c) Was the IV manipulated as independent-groups or within-groups?

d) Which of the four basic experiments does this study seem to have used--Posttest only? Prettest-posttest? Repeated measures? or Concurrent measures?

e) Sketch a graph of the results that were described (hint--put the three IV groups on the x-axis).

f) In the empirical article, the researchers mention that the presentation of the different IV levels was fully counterbalanced, which means that cats in the study were exposed to multiple trials. Why was counterbalancing necessary, and what might counterbalancing have looked like? 

g) The story mentions that "the volunteers wore sunglasses to avoid inadvertently giving the cats any signals with their eyes.". Which internal validity threat was this designed to prevent?

Now here's a detail about the citizen science aspect of the study:

While hundreds of people signed up for the experiment, in the end the researchers got complete data on only 30 cats..... [Smith] notes the number of people who dropped out of this study and says future efforts along these lines may need to be even shorter and easier to ensure that volunteers are able to complete all the required tasks....This latest study, [in contrast to past work on cats] has the advantage of testing untrained cats in a.... natural setting, 

h) Given this outcome, summarize the pros and cons of the citizen science approach to animal research, perhaps adding some of your own.

i) You might be alarmed that the sample size for this study dropped from hundreds of people to only 30, and you might be tempted to think this study has an attrition threat to internal validity. But attrition isn't an issue for internal validity here--can you explain why?