Everyday Research Methods

Saturated fat can impair concentration

Breakfast of (concentration) champions? Maybe not....Photo credit: Kyndall Ramirez/Unsplash

A headline at Psych Central reads, "High fat meal may impair concentration." Does the study behind the headline support that causal claim?

At first, you might assume that this came from a correlational study on diet and concentration. However, it turns out the research summarized in the article was experimental: It manipulated the saturated fat content of one specific meal. In the study, researchers created two versions of a breakfast--one made with saturated fat and the other made with sunflower oil, which is also high in dietary fat but is not the saturated type. 

The journalist describes the method of the study this way: 

Women in the study [n = 52] completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

The high-fat meal followed: eggs, biscuits, turkey sausage and gravy containing 60 grams of fat, either a palmitic acid-based oil high in saturated fat or the lower-saturated-fat sunflower oil. Both meals totaled 930 calories and were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Five hours later, the women took the continuous performance test again.

Between one and four weeks later, they repeated these steps, eating the opposite meal of what they had eaten on the first visit.

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment.

(The original empirical journal article is here; it might be paywalled)

Questions:

a) What was the independent variable (IV) in this study and what were its levels?

b) What was the dependent variable?

c) Was the IV manipulated as independent groups or within-groups?

d) This design is difficult to classify because it has a pretest and posttest, but the women also experienced both levels of the IV. Therefore it shares features with two of the four designs in Chapter 10. What are those two designs it shares features with? (Note to students: As you can see in this example, the four designs you learned about in Chapter 10 are not the only types of experiments there are, and this study provides an example of alternative designs.) 

e) Sketch the results of this study--using "reduction in concentration" (i.e., a difference score) as the dependent variable. 

Next let's work through the four big validities. 

f) Here's some text related to external validity of this study:

For this work, Madison [the lead author] conducted a secondary analysis of data from Kiecolt-Glaser’s study assessing whether high-fat meals increased fatigue and inflammation among cancer survivors.

and

[the experimental meals]  were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Given the information above, to what kinds of people and what kinds of situations might this research be generalized? How do you know? 

g) The researchers found an 11 percent drop in concentration after the meal high in saturated-fat. What further information would you like to know in order to evaluate the study's statistical conclusions? 

h) Can this study support the causal claim that "A high-fat meal may impair concentration"? Apply the three criteria to the study: Covariance (what did the results show?), temporal precedence (did the method ensure that the meal came before the drop in concentration?), and internal validity (What potential confounds did they control for in the study?) 

i) Here's some information about the study's construct validity: 

Women in the study completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

What questions would you need to ask to determine if this test is a good measure of people's ability to concentrate? 

Instructors: If you teach Chapter 12, you can also use this example as a mixed factorial design, because the study also included a participant variable of endotoxemia, a condition called "leaky gut." From the journalist's summary: 

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment. Concentration lapses were also apparent in the women with signs of leaky gut: Their response times were more erratic and they were less able to sustain their attention during the 10-minute test.

“If the women had high levels of endotoxemia, it also wiped out the between-meal differences. They were performing poorly no matter what type of fat they ate,” Madison said.

Students could identify the IV, PV, and DV and practice graphing the results described above. 

Thanks to my colleague Stephen Chew, who shared this example.

Teaching with 100 Humans

In 2020, Netflix released an 8-episode series called "100 Humans" in which 100 (American) humans from a variety of ages and social groups participate in a series of studies. It's a gold mine of examples for critical thinking in research methods. You'll find examples of posttest-only, pretest-posttest designs, HARKing, ethics, and more. Most of the studies have flaws, but that makes them more fun to critique. 

I've cataloged ways you can use the series in your teaching: www.tinyurl.com/teachingwith100humans.

Reach out if you use it! Let me know how it works for you. 

 

What's the deal with predatory journals?

The published study claimed to have observed one parrot, one penguin, and one woodpecker. Photo Credit: Zdenek Machacek/Unsplash

Today's the first time that the National Audubon Society has been featured on this blog! The story is ornithological (sort of) but it's mainly a parable about predatory journals. The journalist for the Audubon Society tells the story of ornithologist Daniel Baldassarre, who published an article in the legitimate-sounding Scientific Journal of Research and Reviews (SJJR). The Audubon journalist explains that there are several odd features to this empirical article. 

The published article, “What’s the Deal with Birds?”, includes the following abstract: 

Many people wonder: what’s the deal with birds? This is a common query. Birds are pretty weird. I mean, they have feathers. WTF? Most other animals don’t have feathers. To investigate this issue, I looked at some birds. I looked at a woodpecker, a parrot, and a penguin. They were all pretty weird! In conclusion, we may never know the deal with birds, but further study is warranted.

The journalist explains:

...the sample size—a woodpecker, a parrot, and a penguin—... seems suspiciously small. The main figure, a graph plotted along an x-axis ranging from “weird beak” to “looks like a fish,” with a red line labeled “the deal,” is a textbook example of dadaist absurdity. The prose veers wildly between academic terminology and gormless observation. “This is the first study I am aware of to attempt to quantify the deal with birds,” Baldassarre writes. “Unfortunately, the results were ambiguous, although Bayesian approaches may prove useful in the future...When presented with weird behavior, birds exhibited a multimodal response including physical aggression and duetting, both of which were repeatable across highly variable contexts.” 

How did such a ridiculous article get published? Turns out that Dr. Baldasarre was trying to prove a point--some scientific journals are not legitimate--they are predatory, and will accept anything.

...the SJRR is a predatory journal, designed to bilk unwary academics out of money, and Baldassarre’s paper is a joke—quite literally—at their expense. 

[...] Baldasarre first had the idea to submit a joke paper in early February, when the latest in a long line of emails from one of these scam journals landed in his inbox. He slapped together the first iteration of “What’s the Deal With Birds”—a couple of paragraphs in the cursory format of a manuscript—only to see it rejected, perhaps as an obvious parody. Undaunted, he inserted some selections from an earlier legitimate paper to pad it out and resubmitted it to SJRR. 

[...]While SJRR initially demanded a $1,700 publication fee, Baldassarre was eventually able to bargain them down to nothing.

You can access the original paper, including its full-color figure, in its predatory-journal-glory here. (Oh no! It looks like SJRR might have taken it down, given all the negative publicity. Here is an independent link.) 

Questions

a) What is a predatory journal and how is it different from a legitimate journal? (Hint: Use the terms peer review and high quality science in your answer.)

b) If you're doing your own library research for a literature review, how can you know if an article you'd like to use is in a predatory journal or not? One way is to ask your librarian. If you are a student, go to your university's library home page, and find where you can access online help from a university librarian. It might be an online chat, a help desk, or an email consultation service. 

c) One potential way to check if a journal is predatory is to use Beall's List. Go there and try typing in the name of the journal (Scientific Journal of Research and Reviews) in this bird story, or try the name of the journal's publisher, Iris Publishers. Does Beall's List consider this journal or its publisher to be predatory? 

d) Visit the homepage of SJRR's publisher, Iris Publishers. Now visit the homepage of two legitimate publishers, Sage Journals and PLOS, both of which publish legitimate science journals. Just by looking at the websites, can you tell which publishers are legitimate?  

e) Predatory journals make money by charging authors (as much as $2000) to publish their articles. But importantly, just because a journal charges the authors for publication does not mean it is predatory. For example, the journal PlosONE is a legitimate journal that charges authors to publish their work. PlosONE is open access, meaning that anybody can access PlosONE articles for free--you don't have to have a subscription via a university library (i.e., the expenses of publication are transferred from libraries to researchers).  Visit the PlosONE webpage and find an example of a (presumably legitimate) psychology article there. 

f) Draw a Venn diagram or concept map depicting the relationships among the following concepts: predatory publishers, legitimate publishers, predatory journals, legitimate journals, authors pay to publish, open-access, peer reviewed.