Everyday Research Methods

Saturated fat can impair concentration

Breakfast of (concentration) champions? Maybe not....Photo credit: Kyndall Ramirez/Unsplash

A headline at Psych Central reads, "High fat meal may impair concentration." Does the study behind the headline support that causal claim?

At first, you might assume that this came from a correlational study on diet and concentration. However, it turns out the research summarized in the article was experimental: It manipulated the saturated fat content of one specific meal. In the study, researchers created two versions of a breakfast--one made with saturated fat and the other made with sunflower oil, which is also high in dietary fat but is not the saturated type. 

The journalist describes the method of the study this way: 

Women in the study [n = 52] completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

The high-fat meal followed: eggs, biscuits, turkey sausage and gravy containing 60 grams of fat, either a palmitic acid-based oil high in saturated fat or the lower-saturated-fat sunflower oil. Both meals totaled 930 calories and were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Five hours later, the women took the continuous performance test again.

Between one and four weeks later, they repeated these steps, eating the opposite meal of what they had eaten on the first visit.

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment.

(The original empirical journal article is here; it might be paywalled)

Questions:

a) What was the independent variable (IV) in this study and what were its levels?

b) What was the dependent variable?

c) Was the IV manipulated as independent groups or within-groups?

d) This design is difficult to classify because it has a pretest and posttest, but the women also experienced both levels of the IV. Therefore it shares features with two of the four designs in Chapter 10. What are those two designs it shares features with? (Note to students: As you can see in this example, the four designs you learned about in Chapter 10 are not the only types of experiments there are, and this study provides an example of alternative designs.) 

e) Sketch the results of this study--using "reduction in concentration" (i.e., a difference score) as the dependent variable. 

Next let's work through the four big validities. 

f) Here's some text related to external validity of this study:

For this work, Madison [the lead author] conducted a secondary analysis of data from Kiecolt-Glaser’s study assessing whether high-fat meals increased fatigue and inflammation among cancer survivors.

and

[the experimental meals]  were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Given the information above, to what kinds of people and what kinds of situations might this research be generalized? How do you know? 

g) The researchers found an 11 percent drop in concentration after the meal high in saturated-fat. What further information would you like to know in order to evaluate the study's statistical conclusions? 

h) Can this study support the causal claim that "A high-fat meal may impair concentration"? Apply the three criteria to the study: Covariance (what did the results show?), temporal precedence (did the method ensure that the meal came before the drop in concentration?), and internal validity (What potential confounds did they control for in the study?) 

i) Here's some information about the study's construct validity: 

Women in the study completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

What questions would you need to ask to determine if this test is a good measure of people's ability to concentrate? 

Instructors: If you teach Chapter 12, you can also use this example as a mixed factorial design, because the study also included a participant variable of endotoxemia, a condition called "leaky gut." From the journalist's summary: 

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment. Concentration lapses were also apparent in the women with signs of leaky gut: Their response times were more erratic and they were less able to sustain their attention during the 10-minute test.

“If the women had high levels of endotoxemia, it also wiped out the between-meal differences. They were performing poorly no matter what type of fat they ate,” Madison said.

Students could identify the IV, PV, and DV and practice graphing the results described above. 

Thanks to my colleague Stephen Chew, who shared this example.

Teaching with 100 Humans

In 2020, Netflix released an 8-episode series called "100 Humans" in which 100 (American) humans from a variety of ages and social groups participate in a series of studies. It's a gold mine of examples for critical thinking in research methods. You'll find examples of posttest-only, pretest-posttest designs, HARKing, ethics, and more. Most of the studies have flaws, but that makes them more fun to critique. 

I've cataloged ways you can use the series in your teaching: www.tinyurl.com/teachingwith100humans.

Reach out if you use it! Let me know how it works for you. 

 

What's the deal with predatory journals?

The published study claimed to have observed one parrot, one penguin, and one woodpecker. Photo Credit: Zdenek Machacek/Unsplash

Today's the first time that the National Audubon Society has been featured on this blog! The story is ornithological (sort of) but it's mainly a parable about predatory journals. The journalist for the Audubon Society tells the story of ornithologist Daniel Baldassarre, who published an article in the legitimate-sounding Scientific Journal of Research and Reviews (SJJR). The Audubon journalist explains that there are several odd features to this empirical article. 

The published article, “What’s the Deal with Birds?”, includes the following abstract: 

Many people wonder: what’s the deal with birds? This is a common query. Birds are pretty weird. I mean, they have feathers. WTF? Most other animals don’t have feathers. To investigate this issue, I looked at some birds. I looked at a woodpecker, a parrot, and a penguin. They were all pretty weird! In conclusion, we may never know the deal with birds, but further study is warranted.

The journalist explains:

...the sample size—a woodpecker, a parrot, and a penguin—... seems suspiciously small. The main figure, a graph plotted along an x-axis ranging from “weird beak” to “looks like a fish,” with a red line labeled “the deal,” is a textbook example of dadaist absurdity. The prose veers wildly between academic terminology and gormless observation. “This is the first study I am aware of to attempt to quantify the deal with birds,” Baldassarre writes. “Unfortunately, the results were ambiguous, although Bayesian approaches may prove useful in the future...When presented with weird behavior, birds exhibited a multimodal response including physical aggression and duetting, both of which were repeatable across highly variable contexts.” 

How did such a ridiculous article get published? Turns out that Dr. Baldasarre was trying to prove a point--some scientific journals are not legitimate--they are predatory, and will accept anything.

...the SJRR is a predatory journal, designed to bilk unwary academics out of money, and Baldassarre’s paper is a joke—quite literally—at their expense. 

[...] Baldasarre first had the idea to submit a joke paper in early February, when the latest in a long line of emails from one of these scam journals landed in his inbox. He slapped together the first iteration of “What’s the Deal With Birds”—a couple of paragraphs in the cursory format of a manuscript—only to see it rejected, perhaps as an obvious parody. Undaunted, he inserted some selections from an earlier legitimate paper to pad it out and resubmitted it to SJRR. 

[...]While SJRR initially demanded a $1,700 publication fee, Baldassarre was eventually able to bargain them down to nothing.

You can access the original paper, including its full-color figure, in its predatory-journal-glory here. 

Questions

a) What is a predatory journal and how is it different from a legitimate journal? (Hint: Use the terms peer review and high quality science in your answer.)

b) If you're doing your own library research for a literature review, how can you know if an article you'd like to use is in a predatory journal or not? One way is to ask your librarian. If you are a student, go to your university's library home page, and find where you can access online help from a university librarian. It might be an online chat, a help desk, or an email consultation service. 

c) One potential way to check if a journal is predatory is to use Beall's List. Go there and try typing in the name of the journal (Scientific Journal of Research and Reviews) in this bird story, or try the name of the journal's publisher, Iris Publishers. Does Beall's List consider this journal or its publisher to be predatory? 

d) Visit the homepage of SJRR's publisher, Iris Publishers. Now visit the homepage of two legitimate publishers, Sage Journals and PLOS, both of which publish legitimate science journals. Just by looking at the websites, can you tell which publishers are legitimate?  

d) Predatory journals make money by charging authors (as much as $2000) to publish their articles. But importantly, just because a journal charges the authors for publication does not mean it is predatory. For example, the journal PlosONE is a legitimate journal that charges authors to publish their work. PlosONE is open access, meaning that anybody can access PlosONE articles for free--you don't have to have a subscription via a university library (i.e., the expenses of publication are transferred from libraries to researchers).  Visit the PlosONE webpage and find an example of a (presumably legitimate) psychology article there. 

e) Draw a Venn diagram depicting the relationships among the following concepts: predatory publishers, legitimate publishers, predatory journals, legitimate journals, authors pay to publish, open-access, peer reviewed. 

Evidence-based tips for well-being during a pandemic

Dr. Santos provides evidence-based ways to enhance well-being during this pandemic. Credit: Yale University

How can people enhance their wellbeing during the Coronovirus pandemic? 

Dr. Laurie Santos (pictured) has been meeting with several news outlets, sharing evidence-based tips from her psychology of well-being class at Yale University. A version of the course is also offered on Coursera.

In the Twitter-embedded video here, produced by the World Economic Forum, Dr. Santos covers five tips for coping during the pandemic. In this space, I'm going to quote moments from the video, and you can practice classifying the claims and describing how the studies would have been conducted. The first three of these tips are also covered in this story from the World Economic Forum.

Tip 1: Keep social, even when socially distancing

Dr. Santos states in her segment that "Happy people spend more time with other people in general. They also prioritize time with their friends and family members"

a) What kind of claim is it to say that "happy people spend more time with other people in general"--frequency, association, or cause?  What are the variables in this claim? 

b) What kind of study was likely conducted to support this claim--correlational or experimental? 

c) Sketch a scatterplot with well-labeled axes that could depict this result.

Tip 2: Help others

To explain this tip, Dr. Santos mentions, "The research shows that doing random acts of kindness...can be incredibly powerful." For example, happy people are giving money to charity and volunteering in their communities. 

d) What kind of claim is it to say that "doing random acts of kindness...can make you happy"--frequency, association, or cause?  What are the variables in this claim? 

e) What kind of study would need to be conducted to support this claim--correlational or experimental? 

f) Can you imagine how you  might manipulate the variable, "doing random acts of kindness?"

Tip 3: Move--even if you can't leave your house

In this section, she says, "Exercise and moving your body is really correlated with improved well-being"

g) The verb, "correlated" gives this one away: It's an association claim. Dr. Santos specifies that they are "really correlated," which seems to suggest a strong effect size. What factors do we consider when we're trying to decide if a correlation (an effect size) is strong versus weak?

Tip 4: Savor everyday pleasures

"Research suggests that happy people tend to be more mindful. They tend to savor the present moment; they tend to be in the present moment"  Here Dr. Santos also strongly recommends taking deep breaths to calm the vagus nerve and connect to the present. 

h) How might you design an experiment to test the hypothesis that deep breathing can improve people's well-being during quarantine? Whom might you recruit for such a study? How would you operationalize each of the two variables? What results would you expect to find? 

Tip 5: Think about how you want to look back on this time

Finally, she shares one of my favorite tips for emotional regulation, psychological distancing: "Psychological distancing is the idea that if we think about ourselves and how we will have dealt with this crisis in the future...Think about how you want to tell your grandkids you lived through COVID-19, you don't want to be like, 'I panicked-bought and hoarded toilet paper..no, you want the story to be positive...'"

Experimental research has supported the causal claim that self-distancing--seeing yourself as though you are a fly on the wall, or even imagining looking back on yourself from the future--is associated with improved emotional regulation. It helps us take a cooler, less impulsive view of our emotions and process them instead of ruminating. For example, self-distancing helps people manage feelings of anger in constructive ways (here's a nice summary). 

i) Review the term translational research in your textbook (Ch 1) and explain how we might conduct translational research on psychological distancing during the pandemic. What is the basic research it is based on? What is the real-world context it's being translated to? And what kind of study could translate--that is, bridge the gap--between the two? 

 

 

 

 

 

 

Random sampling in action for public health

Why is random sampling important when trying to estimate antibody levels in the population? Photo: © Eugene Lu/Shutterstock

Here is a timely example of random sampling in action.

Watch this 2-minute video about the CDC's project in Atlanta (produced by NPR). It explains how nurses are visiting about 500 people in the Atlanta metropolitan area to see how widespread the Coronoavirus is.  In the first minute of the video, the narrator reports,

The CDC is sending out employees to conduct antibody tests on a random sample of Atlanta residents. The tests are meant to show how many people have been infected with the Coronavirus.

The CDC wants to get blood samples from 450 households. 

(Note: And here's another story of a similar effort, this time in Corvallis, Oregon, USA. It's about prevalence of live cases, rather than antibodies, but a random sample is still a key player.)

Questions:

a) Why would it be important to conduct antibody tests on a random sample of people from the Atlanta area?

b) The plan is to collect samples from 450 households. Imagine that a friend hears this figure and expresses skepticism--"how can 450 be enough to generalize to the population of Atlanta?"  Explain to your friend why a random sample--even if it has "only" 450 households--can generalize to the population.

c) What are some ways the CDC might conduct this random sample? Review the possibilities in Chapter 7 and explain:

i) how might they obtain a simple random sample of Atlanta residents?
ii) how might they obtain a cluster sample (what could they use as clusters?)
iii) how might they conduct a stratified random sample? (Imagine that they want to use neighborhoods/zipcodes as the strata.)

 d) After conducting the antibody study on a random sample of Atlanta residents, the CDC will have an estimate of the rate of infection in the sample. To whom will they be able to generalize this rate of infection--that is, to which population of interest?

e) What might happen if some of the randomly sampled people in Atlanta refuse to have their blood drawn for this CDC study? Will that affect the sample's generalizability?  Why or why  not?

f) Which big validity is this example concerning? 

Many thanks to Dr. Jess Harnett (of the notawfulandboring blog), who posted this example, and thanks to Kathy Becker-Blease for sharing the Oregon example. 

Testing a COVID drug: Design flaws and ethical lapses

Photo credit: Alissa Eckert, MS; Dan Higgins, MAMS/CDC

Here's a good example of a bad study. It's related to the COVID pandemic. In this story by NPR  journalist Vanessa Romo, you can read about a doctor in Texas who claims to be conducting research on the drug hydroxychloroquine, using COVID patients in a nursing home as participants. The doctor's research includes both design problems and ethical problems.

According to the journalist, 

...a doctor at a Texas nursing home started giving the anti-malaria drug hydroxychloroquine to dozens of elderly patients diagnosed with COVID-19 and tracking the outcomes in what he's calling an "observational study".

According to the doctor conducting this study, 

"It's actually going well. People are getting better," Armstrong told NPR, adding that after just a handful of days, some of the 39 patients on the medication are showing signs of improvement.

Armstrong admits it is difficult to quantify how much of his elderly patients' improvement is due to the malaria drug or how they would have fared without it. Nor can he explain why other patients are not responding to the tablet doses, though he notes many are only halfway through the five-day cycle.

Design questions

a) Why can't the doctor decide "how they would have fared without it"? Your answer should discuss the study's need for a comparison group. 

b) The doctor's study includes one group--all of the patients in the "study" are taking the drug. What kind of comparison group will be needed to conduct a true clinical trial?

c) If you're as far as Chapter 11, you'll recognize the design of this doctor's study. What is it?

Here's another quote from the article:

[The doctor] said he started by screening those he believed would benefit most and added more people each day. He monitored their blood oxygen saturation, temperatures and how well they were breathing. "The people who [I decided to put] on it were getting sicker but were not so sick that they had to go the hospital," Armstrong explained.

d) What design problems, if any, to you see with the doctor selecting only this type of patient for being in his "observational study?" And, what might have happened if he had included more severely ill patients for his study? (How might the results change?)

Ethical questions

According to federal guidelines and the ethical principles of the Belmont Report, when a person claims to be doing "research," they should obtain ethical approval from an IRB. Ethical approval would require such procedures as obtaining informed consent (respect for persons), minimizing harm (beneficence), and selecting participants fairly (justice). Read these next quotes from the story:

He acknowledged that some families were not aware their relatives were put on the drug, saying that "for the most part," he consulted with each nursing home resident prior to giving them on the tablets.

While the "overwhelming majority of them are awake and alert and can actually have a conversation," Armstrong said some suffer from middle stages of dementia. In some cases, he did not discuss prescribing the tablets with anyone at all before doing so. He said it is common for physicians to prescribe new medications to patients without explicit consent from the patient or family members. "It's not required," he said.

e) Review the sections of Chapter 4 on "respect for persons," "beneficence," and "justice," and apply details from these concepts to this example. 

f) Do you think study design flaws such as the ones you've noted in a), b) and c) should be considered ethical violations, too? Why or why not? 

Safety questions

A potential issue with this doctor's decision concerns the safety of the drug hydroxychloroquine: According to the journalist:

The Food and Drug Administration has not approved the drug for the treatment of COVID-19. 
and
[the drug] is known to have serious negative health impacts. That is why so many in the medical community worry about prescribing it without such proof.

g) The safety of the drug is relevant to the ethical principle of beneficence. Can you explain why?

Finally...

Another wider ethical issue (perhaps outside the scope of a research methods course) concerns how the doctor obtained the drug, which has been in short supply:

[The doctor] began making calls to track down a source for the medicine, which is in short supply.That's when his political connections proved useful.

Armstrong, who is a prominent GOP activist, called Republican Lt. Gov. Dan Patrick. He says Patrick reached out to Texas state Sen. Bryan Hughes, also a Republican, who knew someone on the board of the New Jersey-based company Amneal Pharmaceuticals. The company, which makes and distributes the drug, has donated more than a million tablets nationwide, including to the statesof Texas and Louisiana.Two days later, Armstrong had received more than enough medication to begin giving it to patients. 

h) Do you think it is reasonable for a doctor to use political connections to obtain medications that are in restricted supply? Why or why not? 

Suggested answers to selected questions

a) and b) and c) Because the doctor's "study" has only a group of patients who are taking the drug, he doesn't have a comparison group. In Chapter 11 this is called a one-group, pretest-posttest design, otherwise known as the Really Bad Experiment. The design is flawed. If patients improve on the drug, the doctor will not know whether their improvement was caused by the drug itself, or simply because their bodies were healing or because they were getting excellent nursing care, or even some other reason. 

The doctor's behavior in this COVID study reminds me of the Halstead radical mastectomy, which for decades was the dominant treatment for breast cancer in the U.S.. Surgeons at the time were so convinced that this treatment was the best one for breast cancer that they never bothered to compare it to another treatment--they never tested its efficacy. (When a randomized control study was finally conducted in the 1970s, it found that the surgery was no more effective than other, less invasive treatments.)

d) By selecting only relatively less sick patients, the doctor is stacking the deck in favor of finding evidence that the drug is effective. Many of the patients are likely to get better, so the doctor may conclude that the drug has worked. But of course, without a comparison group of equally healthy people who are not getting the drug, we cannot know if the patients would have recovered anyway.

e) f) g) The lack of informed consent--especially for vulnerable patients who have Alzheimer's--is a clear violation of the respect for persons principle. And the use of a risky treatment, one that is known to be dangerous to people with heart conditions, is likely a violation of the beneficence principle. I would argue that when a researcher conducts a flawed study like this, it also a violation of beneficence--the study puts participants at risk, while the benefit of the potential information gained from the research is negligible. We might also argue that the study violates justice: by studying only people at a nursing home, these patients bear the burden of being research participation when anyone can benefit from this research (assuming, of course, that the research were actually well-designed).

Many thanks to Dr. Stephen Chew for pointing me to this example!

Post script: Looking for more examples of bad studies, as described by science writers? Here are two:

Here's a critique of the many problems with another early published study on hydroxychloroquine.

Here's critique of studies supposedly investigating the death rate of COVID-19. 

Temperature data as quasi-experiment

Which government-led social distancing measures are effective at stopping the spread of the novel Coronovirus?  One company has contributed data to answering this question. A private health technology company, Kinsa sells a "smart thermometer" which links to an app that guides people through what to do if they have a fever. When people use the app, they agree to have their temperature information (including fever and other symptoms they report) shared with Kinsa. 

Kinsa has used body temperature data points from all over the U.S. to create maps showing areas of "atypical illness." They define atypical illness on their website as "an unusual incidence of elevated flu-like illness levels."

It's really interesting to explore Kinsa's atypical illness maps of the U.S. here. 

More relevant for a research-methods course are the figures that Kinsa presents that map "flu-like illness" rates over time in certain high-population counties (Boston, New York, Los Angeles). I've pasted two of them here. Take a look at these figures, and then answer the questions below. 

Image courtesy of Kinsa, Inc.

 

Image courtesy of Kinsa, Inc

 

a) These figures should remind you of one of the quasi-experimental designs covered in Chapter 13. Which design is it?

b) The Kinsa company has used these charts to argue that social distancing measures work to reduce cases of COVID-19. Journalists (for example) have used the charts to specifically argue that limiting large gatherings and closing schools alone does not do much to stop the spread; it was only when bars and restaurants were shuttered that these counties saw reduced rates of atypical illness. Look carefully at the three counties above. What parts of the data are relevant to this argument? Do you think the data support the claim that closing bars and restaurants was effective? Can you think of alternative explanations for the pattern, other than "closing bars and restaurants"?

c)  Finally, consider the ethics and scientific openness principles. Do you think it's ethical for Kinsa to use and publicize the private health information of people who buy its products? Consider arguments both for and against the scientific ethics of this practice. Frame your arguments around the ethical principles of respect for persons, beneficence, and justice. 

In your reflections, you might consider that users of Kinsa's products agree to data sharing via its terms and conditions. You might consider that the data from users is kept anonymous and is aggregated over more than a million users. You might consider the public health benefits of sharing the data. And you might also consider the extent to which Kinsa is following open science practices--that is, sharing all the data the way most university researchers now do (relevant Kinsa research information here). 

d) If you're construct-validity-curious, you might be wondering if Kinsa's definition of flu-like illness actually tracks COVID-19 cases. Kinsa do not claim to be diagnosing actual COVID (you need a nasal swab for that). However, they present some maps showing how their temperature data has tracked with confirmed COVID cases in several counties. The data are in the middle of this webpage.  

By the way, here's another example of a company that collects our health data, using it to share national patterns. This one is Fitbit, sharing how sleep time has increased (in most age groups) since the COVID quarantine started.

 

Is "sugar high" a thing?

 

Sugar doesn't make kids hyperactive; at least not according to a null effect study and a meta-analysis. Photo: Alamy

Here's something "everybody says":  Kids will become hyperactive if they eat too much sugar. This folk belief is as much a part of North American cultural wisdom as learning styles and the catharsis hypothesis. But this column in the New York Times explains that, like learning styles and catharsis, the idea of a "sugar high" is also a myth. 

 

The physician who wrote the column traced the modern idea of sugar causing hyperactivity to 1975. But then the author describes a randomized clinical trial on this question. Here's the description: 

An extraordinarily rigorous study settled the question in 1994. Writing in the New England Journal of Medicine, a group of scientists tested normal preschoolers and children whose parents described them as being sensitive to sugar. Neither the parents, the children nor the research staff knew which of the children were getting sugary foods and which were getting a diet sweetened with aspartame and other artificial sweeteners. Urine was tested to verify compliance with the diets. Nine different measures of cognitive and behavioral performance were assessed, with measurements taken at five-second intervals.

The original study was, indeed, ambitious. The authors provided all food to the homes of the children in the study so they could completely control what sweeteners were being used. The study took place over nine weeks in a repeated-measures design--the diets were based on sucrose (sugar) one week, aspartame another week, and saccharine during another week. The researchers measured the children's cognitive performance and mood throughout the study.

The study concluded that sugar does not affect children’s behavior or cognitive function. An editorial that accompanied the study put a fine point on that conclusion, stating “there is no evidence that sugar alone can turn a child with normal attention into a hyperactive child.” 

Questions

a) What were the independent and dependent variables in the study described here? What are the levels of the IV?

b) The IV was manipulated as within subjects, using a repeated-measures design. Describe why counterbalancing would be important here, and how you could counterbalance in this study. Which of the four big validities does counterbalancing support?

c) What questions could you ask about the other three validities for this study? 

d) This study found a null effect (no impact of sucrose on children's behavior). There were only 48 children in the diet study, and this might be considered a low number--there may not have been enough kids in the study to detect a small effect of sucrose. However, the repeated measures design means this study can make a stronger case for the conclusion that sugar doesn't affect kids' behavior. Review the information in Chapter 11 to explain why the repeated measures design helps support the conclusion. 

The author of the piece mentions this, too:

One year later, an analysis that gathered data from all published studies on the subject reached the same conclusion.

e) This sentence is describing a meta-analysis. In your own words, describe what steps they would have taken to conduct this meta-analysis.

f) Why is a meta-analysis stronger evidence for a null effect of sugar on hyperactivity? 

g) Given this information, what would you say to a relative or friend who believes that sugar causes hyperactivity in their own child? 

 

A serendipitous study of long-term memory in birds

Here's a photo of a toutouwai (North Island Robin) sporting its bands for identification. Photo: Shutterstock

We don't include many animal behavior studies in ERM blog, but here's a great one about animal memory. How long do you think wild animals can remember new skills they have learned? In this report in The Scientist, we read that the North Island Robin (known in the Maori language as toutouwai) of New Zealand can apparently remember a skill up to 22 months, without any reminders in the meantime. Here's how the study worked. 

[Originally, scientist Rachel Shaw] demonstrated that wild toutouwai living in the Zealandia Wildlife Sanctuary in Wellington, New Zealand, are capable of learning a novel task. Using a rectangular box with three compartments, each covered by a lid that swivels to the side to reveal its contents, she incrementally taught the birds how to peck the lids aside and retrieve mealworms hidden inside two of the three wells.

(By the way, the story in The Scientist includes a photo of the experimental apparatus and a video of a bird using it.)

Roughly a year later, Shaw was giving a public talk at the sanctuary about conservation and her work with birds. As she explained the memory apparatus she had used in her previous research, a toutouwai appeared.

“I haven’t given this bird this task in over a year,” she recalls telling the audience, “so let’s see how he does.” As if on cue, the bird instantly flew down to the apparatus and started opening the lids. That got her wondering: how many of the other birds could repeat the memory task and how long could they remember what they were taught?

This serendipitous event prompted Dr. Shaw to design a study to find out. Before reading on, how would you study this question? 

In the new study, Shaw and her coauthor, Annette Harvey, tested 32 of the initial 64 birds that were trained in 2015 and 2016 and banded for individual identification, 30 of which performed the memory task by spontaneously pecking the lids and opening them on their first try. None of the trained birds had seen the box between the initial training and retesting, though some of the untrained control birds had encountered the apparatus in a previous research project. The time between when the birds had first learned the lid-opening behavior and subsequent testing ranged from 10 to 22 months. By contrast, the 17 untrained birds were unable to complete the task.

The article mentions several other examples of long-term memory in animals, including pigeons, tortoises, nutcrackers, and chickadees. Read more if you are interested!

Questions

a)  This study might be considered a simple observational study but it might also be classified as a quasi-experiment. What do you think?

b) What role does the untrained comparison group play in this research? 

c)  Do you think the effect size in this study would be considered small or large? Explain your answer.

d) Chapter 6 describes important construct validity considerations for observational studies, such as reactivity, observer bias, and observer effects. Which of these might be relevant in this study on toutouwai? 

Thanks to my friend Dr. Stephen Chew for sharing this story on Twitter!

Claim: "Going to museums and concerts helps you live longer"

Does the study's method support the claim that attending arts events makes you live longer? Photo credit: agefotostock/Alamy

We'll never run out of material in the "correlation is not causation" department. 

This time, a New York Times  journalist led with the headline, "Another benefit of going to museums? You may live longer." 

Let's check out the study behind this causal claim. 

According to the journalist's summary of the empirical article, 

Researchers in London who followed thousands of people 50 and older over a 14-year period discovered that those who went to a museum or attended a concert just once or twice a year were 14 percent less likely to die during that period than those who didn’t.

The chances of living longer only went up the more frequently people engaged with the arts, according to the study, which was published this month in The BMJ, formerly The British Medical Journal. People who went to a museum or the theater once a month or even every few months had a 31 percent reduced risk of dying in that period, according to the study.

a) Starting just from here--what are the two variables they've mentioned so far? Were they manipulated or measured? 

b) Based on what you've read so far, is this study correlational or experimental? 

c) Without reading further, think of a third variable (a "C") that is likely to be correlated with both A (going to museums or concerts) and B (being less likely to die). Any good ideas? 

OK! Time for more information about the study from the journalist:

The study controlled for socioeconomic factors like a participant’s income, education level and mobility, said Andrew Steptoe, a co-author of the study and the head of University College London’s research department of behavioral science and health.

“Even if you take those things into account, we still found that there is a difference in the survival of people who are involved in the arts,” Professor Steptoe said in an interview on Friday.

 

Sounds like multiple regression was used here.

d) What was the criterion (dependent) variable in this study now?

e) What were the predictor ("independent") variables? 

f) Sketch a little regression table similar to those in Chapter 9--put the criterion variable at the top and the predictors at the bottom. Which betas do you think were significant?

g) More importantly, was the "C" variable you thought about in c), above, controlled in their regression?

h) Even though the authors controlled for income, education, and mobility, there are likely to be additional third variables that were NOT controlled for. Think of at least one more alternative explanation. 

 

As you learn in Chapter 9, regression does not support causation because even though it can control for alternative explanations you can think of (such as income, education, or mobility), it doesn't control for everything. If you read the Time's summary, you'll see even more variables they seem to have controlled for , such as marital status and profession. But there could still be unmeasured variables that are responsible for the relationship. 

 

But that doesn't stop the authors from making causal speculations. In fact, here's a mediator hypothesis. i) Sketch this mediator following the diagrams at the end of Chapter 9:

 

The study also noted that engaging in the arts can reduce loneliness, promote empathy and emotional intelligence, and keep people from becoming sedentary — all factors that contribute to a longer life.

 

What kind of study would you need to conduct to test this proposed mediator?