Everyday Research Methods

Multivitamins and memory in older adults

Participants in the study took either a multivitamin or a placebo.
Photo: Sorapop Udomsri/Shutterstock

When you shop in a natural foods or general nutrition store, you might see claims that vitamins can improve your health and cognitive function. Scientists don't always test such claims in randomized trials, so it's best to be skeptical until you read research. But a recent study did test whether taking a multivitamin can improve people's memory. In this blog post, we'll check out how an NPR journalist summarized the study.

Before reading on, think about this: How would you design a study to test this question? What would be your independent and dependent variables? What kind of study would you use--specifically, what type of experiment (Posttest only? Pretest/posttest? Concurrent measures? Repeated measures?)

Here's the journalist's summary of the study:

A team of researchers wanted to assess how a daily multivitamin may influence cognitive aging and memory. They tracked about 3,500 older adults who were enrolled in a randomized controlled trial. One group of participants took a placebo, and another group took a Silver Centrum multivitamin, for three years. The participants also took tests, administered online, to evaluate memory.

At the end of the first year, people taking a multivitamin showed improvements in the ability to recall words. Participants were given lists of words — some related, some not — and asked to remember as many as possible. (List-learning tests assess a person's ability to store and retrieve information.)

People taking the multivitamin were able to recall about a quarter more words, which translates into remembering just a few more words, compared to the placebo group.

a) Make a table of the variables in this study:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) This was an experiment. How do you know--what are some key words used in the journalist's description?

c) Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

d) Sketch a graph of the study's results. Here's a hint: You should put time (or pretest/posttest) on the x-axis). 

Next we'll apply the four big validities to this study. 

Construct validity
e) Ask a question about the construct validity of the dependent variable.  What do you need to know in order to assess construct validity of the DV?

Statistical validity
f) One of the questions to ask is about effect size: How much better was the memory in the multivitamin group?  The NPR report offered two quotes related to effect size.  One is from the study's own authors: "'We estimate that the effect of the multivitamin intervention improved memory performance above placebo by the equivalent of 3.1 years of age-related memory change,' the authors write in their paper"

The other is from an independent researcher, who "says the overall effect found in the study was quite small. 'It seems like a pretty modest difference'" 

Do these two people disagree about the study's effect size? 

g) Another aspect of statistical validity is precision. The sample size of the study is relevant here--why? What effect would a large sample (of over 3,000 participants) have on the confidence interval of the estimate?

h) A third aspect of statistical validity is replication, or what else is known? Here's a relevant quote: "Study author Dr. JoAnn Manson...says this is not the first study to show benefits from multivitamins. She points to a study published last year in Alzheimer's & Dementia that showed participants who took a daily multivitamin performed better, overall, on global cognitive function on tests measuring story recall, verbal fluency, digit ordering, as well as executive function."

External validity
i) What do you need to know to evaluate whether the results of this study can generalize to, say, older adults in North America? (check your answer--you did not talk about sample size, did you?)

Internal validity
j) This study is described as a "double-blind, randomized, placebo-controlled" design. Knowing this, you can rule out many internal validity threats. Briefly explain why the study allows you to rule out:

• Selection effects
• Placebo effects
• Testing threat
• Maturation
• History
• Observer bias

At the end of the article, NPR reported

The study was funded, in part, by the National Institutes of Health and other grants. The vitamins were provided by Pfizer Inc. and Haleon, the makers of Centrum, the brand of multivitamins taken by participants in the study. The study authors say the funders had "no role" in the study design, analysis or interpretation.

Why is this statement important for observer bias? 

 

 

 

 

Does cannabis make people more creative?

To measure creativity in the study, participants came up with creative uses for a brick. Photo: levkro/Deposit photos

A lot of people believe that using marijuana can make them more creative--some artists use it specifically when they are composing songs, painting, or creating new work. 

A recent study experimentally tested this idea. The study provides a useful example of a null effect. Here is a summary by a journalist in the Washington Post. 

The journalist writes:

...researchers designed a randomized controlled trial comparing the creative outputs of light cannabis users who had just imbibed vs. those who had not.

In a first set of experiments, about 107 light users of cannabis were asked to take a standard creativity test within 15 minutes of getting high. For the control group, the researchers asked 84 other participants to do the task only if they had not used cannabis in the past 12 hours, by when any effects would have worn off.

For the creativity test, the participants were asked to consider a brick, and generate as many creative uses as possible for it in four minutes. As expected, participants who were under the influence felt more jovial, which made them feel like their ideas were more creative than sober participants.

But creativity assessments by third-party raters, who did not know who was under the influence, were a buzzkill. When they rated the responses on novelty and usefulness, they did not see a difference in creativity.

Questions

a) Start by making a table of the variables in this study.

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) Check your work above. There were two operationalizations of the dependent variable, creativity. Do you have both of them?  Do you also have the dependent variable of joviality? 

c) Was this study an experiment or a correlational study? How do you know?

d) Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

e) Sketch two graphs of the study's results: One graph for each dependent variable. (Hint: The x-axis will show the same independent variable in both graphs). 

f) The results which showed that, when creativity was measured by third-party raters, there was no effect of using cannabis on creativity. This is a "null effect" as introduced in Chapter 11. Flip to Table 11.3 in your text: we'll apply several of the questions to this study. 

• Could there have been an ineffective manipulation of the IV? 
  
  
• Is there evidence of a ceiling or floor effect on the IV or DV?
  (Hint: the fact that there were differences in joviality and self-reported creativity suggest that the IV actually did work--right?)
  
  
• What are some potential causes of within-groups variability in this study? How might the researchers have minimized their effects?  Consider situation noise, measurement error, and individual differences.