Everyday Research Methods

What behaviors make you popular? What behaviors make you likable?

The study investigated two types of behaviors and two types of liking. Photo credit: Rawpixel.com/Shutterstock

Can psychological science make you more popular?  That's what a journalist from Forbes claims in an article, "A science-backed way to up your popularity and friendship." Let's see what the journalist is claiming and see if we buy the evidence. 

The journalist's summary focuses on an empirical article published in Psychological Science (may be paywalled). 

The researchers started with a theory about...

...two types of behavior that can lead to liking during a first encounter:

1. 1. Agentic behavior is usually exhibited when one wants to gain someone’s respect or admiration. Examples include dominant or confident behavior.
   2. Communal behavior, on the other hand, is shown when the goal is to form mutually supportive and trusting bonds with someone. Examples include warm, welcoming, and friendly behavior.

Both agentic and communal behavior have been studied in other social psychological research.

Here's how the study was conducted: 

The study recorded brief, first encounter conversations between two participant groups such that each member of one group interacted with each member of the other.

Specifically, each member of a small group had a 5 minute introductory conversation with each other member of the group. After everybody had talked to everyone, each member of the group rated how much they liked each other person. 

Importantly, the journalist explained that the researchers  were interested in two types of liking: 

1. Popularity, i.e., how much is an individual generally liked by others
2. Unique liking, i.e., how much an individual is particularly liked by a specific person

a) Now that you've got this background, make a prediction in your head. Which kind of behavior--agentic or communal--do you think will be most associated with popularity? Which kind of behavior will be most associated with unique liking? 

Here's how the journalist summarized the study's results:

The study produced two key findings:

1. 1. Showing frequent agentic and communal behavior can lead to high popularity.
   2. When it came to unique liking, exhibiting high levels of communal, but not agentic, behavior led to being particularly well-liked by the interaction partner, more than they liked their other partners as well as more than their other interaction partners liked them on average.

The Forbes journalist doesn't tell us much about how agentic and communal behaviors were measured in this study. Therefore, I turned to the original empirical journal article to find out. It turns out that they had a separate group of raters watch videos of the interactions, rating each participant's behaviors. Here's a passage from the study's Method section:

After the study was complete, independent observers watched the video material and rated the degree to which each participant showed agentic or communal behaviors toward each other group member during the getting-acquainted conversations. (Observers saw videos showing both interaction partners but were told to focus on one of them.)

A rating manual provided behavioral anchors of the four agentic and the four communal behaviors so that raters would have a common understanding of what these behaviors meant. As agentic behaviors, the observers indicated how “leading,” “dominant,” “confident,” and “boastful” a participant’s behavior was. As communal behaviors, the observers indicated how “polite,” “benevolent,” “warm,” and “friendly” a participant was .... Ratings were made on a 6-point Likert scale ranging from 1 (not at all) to 6 (very much).

b) In Chapter 5, you learn about three types of dependent measures: self-report, observational, and physiological. This is a clear example of an observational measure. Which of the three types of reliability is relevant for an observational measure? 

Here is another passage from the empirical journal article: 

The observer ratings demonstrated moderate to good interrater agreement for all eight interpersonal behaviors—the intraclass correlation coefficients(2, k) ranged from .66 to .84. 

c) Imagine there are only two coders....could you graph the result of the passage above, following models in Chapter 5? Pick one of the ratings to focus on. 

d) This was a correlational study, not an experimental one, which means that the journalist probably jumped the gun when he suggested "Showing frequent agentic and communal behavior can lead to high popularity."  How might you change this study be experimental? Specifically which variables would you have to manipulate? Which would you measure? 

Do TikTok dance videos influence body perceptions?

Seeing different size models on TikTok videos affected people's body esteem.
Photo: AlessandroBiascioli/Shutterstock

Have you ever watched dance challenge videos on TikTok? If so, you probably noticed the skill and appearance of the dance model.  Some psychologists have started to study how TikTok dance models affect viewers.  A set of three studies asked, can the size of the dancer on TikTok influence the body satisfaction of a viewer?  Journalists at PsyPost wrote up a summary of this empirical work. The research was originally published in the journal, Computers in Human Behavior; may be paywalled. 

The theory behind the study is related to the potential of social media to impact body esteem:

Media, especially its portrayal of the “thin ideal,” where extremely slim bodies are considered attractive, has a significant impact on shaping these negative body perceptions. With the rise of social media platforms like TikTok, researchers are curious about how this new form of media might influence people’s body image.

Here's a description of the first study. As you read this description, try to identify the variables in the study:

...the researchers recruited 262 women aged between 18 and 25 years for an online experiment.  The participants were randomly assigned to watch TikTok videos featuring thin dancers, large dancers, or videos of animals (the control condition). [Both before and after] watching the videos, participants rated their weight satisfaction, overall appearance satisfaction, and body shape satisfaction.

Body satisfaction significantly increased after watching videos featuring large dancers and control videos (videos unrelated to body appearance). However, after watching videos of thin dancers, body satisfaction showed a slight decrease.

a) Use this chart to identify Study 1's variables (Hint: There are one IV and three DVs)

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? How do you know?

c) Sketch a bar graph or line graph of the results they described (select one of the three DVs). Don't forget to plot both pretest and postest data. 

The researchers did Study 2 to replicate Study 1:

Study 2 largely replicated the findings from Study 1. Watching videos featuring large dancers led to increased body satisfaction, while watching videos featuring thin dancers resulted in decreased body satisfaction.

Study 2 also measured the potential mechanism, or mediator, of social comparison (how often someone compares their appearance to those of others.).  Study 2 found: 

A majority of participants in the thin dancer condition engaged in upward comparison (comparing themselves negatively to the dancers), while a majority of participants in the large dancer condition engaged in downward comparison (comparing themselves positively to the dancers).

One potential internal validity threat to both Study 1 and Study 2 is that participants were told the study was about body satisfaction. In addition, the pretest-postest nature of the design may have nudged participants to report changes in line with the study's goals.

Study 3 attempted to correct this issue with a posttest-only design, and by telling participants that the study was actually about how engaging different TikTok videos are. 

d) What is the name for this type of internal validity threat? 

e) Why would a posttest-only design help control for this internal validity threat?

Here's the journalist's summary:

In Study 3, the researchers recruited 375 women aged between 18 and 25 years and randomly assigned them to watch either TikTok videos of thin dancers or TikTok videos of large dancers. Participants were told that the study aimed to investigate the engagement of TikTok videos. After watching the videos, the participants completed the same measures of body satisfaction used in the previous studies.

Even after concealing the true purpose of the study, the results consistently indicated that watching videos of thin dancers led to lower body satisfaction compared to videos of large dancers.

f) Reread the summary of Study 3 and identify its variables on the table below:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

g) Graph the results of one of the dependent variables in Study 3. 

h) What do you think of this set of studies? Ask questions about the four aspects of validity.

Specifically, when you address statistical validity, use terms like replication and effect size.
When you address internal validity, think about how the study rules out most internal validity threats.
When you address construct validity, ask a question about the measurement of the dependent variables.
When you address external validity, think about the samples used in these studies.

Here's how the researchers concluded an interview about their studies:

“I would like to add that this study shows that watching TikTok videos is not universally negative and that there are positive and negative effects,” Joiner said. “The findings also raise the possibility that TikTok could change its algorithm and show a wider variety of body types, and this could have a major public health benefits. Body dissatisfaction is major risk factor for eating disorders and eating disorders have the highest mortality rate of all psychiatric diseases due to medical complications or suicide.”

Easing the angst of upsetting news

The study tested two different types of stories to see which might ameliorate the stress of reading about tragic news events. Photo credit: Nomad_Soul/Shutterstock

This post contains descriptions of violence in the news.

Have you ever been saddened or dismayed by a news about a terrible event, such as a natural disaster, mass shooting, or a crime? You're not alone; such stories can make us emotional. One psychology researcher, Kathryn Buchanan, was inspired to study people's reactions to news events by her own experience. She was listening to news about the mass shooting at the Ariana Grande concert in 2017. She felt upset at the news, but she also felt some relief, when she later encountered news about examples of people's kindness in the shooting's aftermath. 

This journalist's account in the Washington Post describes how Dr. Buchanan turned her experience into a research idea:

She began to contemplate whether being exposed to heartwarming content could counteract the known negative impacts of consuming harrowing news stories. Common symptoms include heightened stress, hopelessness, anger, anxiety and depression.

So she started a years-long study in 2017,[...]. [The research team] split 1,800 participants into several groups — each of which were shown news clips or articles about a recent U.K.-based terrorist attack or a similar catastrophic event. Some of the groups were solely exposed to upsetting news, while others were subsequently shown a story about [either] a heroic act or a more lighthearted subject.

Here is a description of some of the results:

The group that was strictly shown unsettling news reported increases in negative emotions and decreases in positive feelings. That group also indicated feeling more pessimism about humanity and society. Meanwhile, participants who were also shown stories of kindness reported feeling less negative emotions, coupled with increases of positive emotion. That group reported a more optimistic view of the world.

Participants used two scales from 1 (meaning very slightly or not at all) to 5 (meaning extremely) to assess the level of negativity and positivity they felt before being exposed to the content, as well as after.

Repeatedly throughout their research, Buchanan and Sandstrom saw that uplifting news can provide an emotional buffer against distressing news. They also found that “there’s something special about kindness in particular,” Buchanan said, noting that while amusing stories (such as swearing parrots or classic jokes) diminished the effects of upsetting news, stories about acts of kindness were even more powerful.

a). See if you can classify the variables in the study. Here's a hint: There were four dependent variables and one independent variable. 

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? How do you know?

c) Look at the description of the results, above, for level of positivity only. Sketch a graph of the results they described. Here's a hint: it's important to find a way to include the pretest and posttest scores on positivity. 

d) Think about the four big validities and apply them to this study. Specifically, ask one question to help you interrogate external, internal, construct, and statistical validity.

I appreciated that, in the news story, the journalist mentioned other studies that also measured people's reactions to negative news in the media. Journalists do not often talk about recent studies in the context of an entire literature, but this one did! You can read examples of conceptual replications of this study in the journalist's story (scroll toward the end).  

The empirical study can be read for free here at PLOS One. 

 

Culture and the "marshmallow test"

Cultural practices around both snacks and gift-giving were proposed as one of the mechanisms behind this study's results. Photo credit: Prostock-studio/Shutterstock

In a recent study, researcher Yuko Munakata tested the self-regulation and self-control abilities of children in two cultures: The U.S. and Japan. She published the research in the empirical journal Psychological Science, but also wrote an accessible description of it for the popular online magazine, Scientific American. The heading wrote, A classic test of self-control can carry complex cultural biases.

In case you are unfamiliar with the marshmallow test, here is Dr. Munakata's description of it:

The classic marshmallow test, invented by the late psychologist Walter Mischel, involves presenting a child with one marshmallow and explaining that they can have it now or they can have two later if they wait until the marshmallow giver returns. Children sit alone in a room with the tempting treat. Psychologists have generally viewed the length of time kids resist the sweet as a measure of their self-control: how effectively they can inhibit impulsive behaviors and work toward longer-term goals.

In their study, the research team compared marshmallow waiting time among four- and five-year-old children in two cultures. Here's a description of part of the study:

We worked with 144 children in both the U.S. and Japan. We first ensured that all of these four- and five-year-olds had eaten marshmallows before. We asked parents how often their children waited to eat until others were served and how well their children could suppress impulsive behaviors.

Then the kids faced the classic test: Did they want one marshmallow now or two later? Most children in the U.S. waited less than four minutes before tasting the one marshmallow. Most children in Japan waited for two marshmallows for the maximum possible time—15 minutes!

But the study did not stop there:  

If we had stopped there, we might have simply concluded that Japanese kids have better self-control. But we conducted another test. We presented children with a wrapped gift and told them that they could open it now or they could have two gifts if they waited. The pattern flipped. Most children in Japan waited less than five minutes before unwrapping the gift, while most children in the U.S. waited the maximum 15 minutes or close to it.

In a moment, we'll go over the researchers' reasoning about why they found this cultural difference. But for now, see if you can identify the features of this factorial design. 

a) Take a moment to classify the variables. There are three of them!

Variable name	What are the variable's levels?	Is this manipulated or measured?	Is this an IV, PV*, or a DV?	For IVs and PVs: is it within groups or independent groups?

*PV means participant variable--it is treated as an IV, but it is not actually manipulated. Examples of PVs are demographic variables such as gender, ethnicity, or class year. 

b) This was a factorial design. What kind? Your answer should read ___ x ___, and mention whether it was an independent groups factorial, a within-groups factorial, or a mixed factorial design (hint: to decide, look back at how you coded the IVs and PVs on the table. 

c) See if you can sketch the results the author described. You can choose a bar graph or a line graph. Remember to put the DV on the y-axis, one IV/PV on the x-axis, and the other IV/PV as a line or bar color. 

d) Do you think this factorial design's results found an interaction? If so, describe it in a sentence or two. 

Now let's read the researcher's explanation for why they that cultural differences in self-control depended on the task used:

What was going on? It seems that with repeated experiences delaying gratification, children can develop habits that make it easier to wait in the future. According to the reports from their parents, the Japanese children had stronger habits of waiting to eat than the U.S. children. The stronger these habits, the longer the children waited for two marshmallows. But when it comes to opening gifts, children in the U.S. likely experience waiting more consistently. Birthday presents can sit on a table, unopened, until the end of a party. Christmas presents may sit under a tree for days before they can be unwrapped on December 25. In Japan, people give gifts year-round on simple occasions that do not involve traditions of waiting.

This result was fascinating to me as a cultural psychologist! Turns out there was no "culture main effect"--there's no overall difference in self control between U.S. and Japanese children. Instead, culture interacts with task, so kids do better at the self-control task they practice more in their culture.

 

"Want to raise kind, generous kids? Take them to an art museum"

Did the study manipulate exposure to art museums?
Photo: AChubykin/Deposit Photos

On my first read of this headline, I was skeptical. "Probably a correlational study behind this one," I thought. As it turns out we are actually looking at an experiment. However, there's another problem with the journalist's coverage of this study. Read on.

The journalist writes for Inc.com, which  summarizes research for the business world. The headline was "Want to raise kind, generous kids? Take them to an art museum" The original empirical article appeared in Psychological Science. Let's work through it. The introduction starts out:

Everyone wants to raise successful kids, but a big part of being successful is being a kind and generous person. So how can parents give their kids the best while also doing their best to ensure they raise decent, thoughtful little people? 

They introduce the idea that art might make kids more generous: 

You may be wondering what possible connection a day viewing Van Gogh or Picasso could have to raising an empathetic kid. Certainly, museums make children more worldly and cultured, but how do they make them kinder? The link, according to the new study, recently published in Psychological Science, is awe. 

a) Before you read more, make a prediction. Given this introduction, what independent variable do you think the researchers probably manipulated? What variable do you think was measured?

Now let's read the details. There were two studies. In the first one, 

..the research team asked 159 volunteers aged 8 to 13 to watch short movie clips. Some of these clips were neutral, others cheerful, and others awe-inspiring. . 

In the second study, about 350 kids participated. The researchers used one of the same three clips for the awe variable (neutral, joy, and awe), but in this case, the generosity variable was operationalized differently:

The researchers then asked the kids [in Study 1] to determine how many items in a list of foods should be donated to a food drive for needy families. Alternately [in Study 2], the kids were given the option of donating their reward for participating in the study--a ticket to a local art museum--to a refugee family. 

(By the way, the original article reports that "The awe-eliciting clip was from the film Song of the Sea, in which the main character turns into a seal and transforms the city while flying above it.")

Back to the Inc.com story for the results: 

"Children who watched the awe-inspiring video chose to count 50 percent more items for the food drive than children who watched the joy-inspiring clip and more than twice as many items as children who watched the neutral clip. [in Study 2], children in the awe-inspiring condition were also 2 to 3 times more likely to donate their museum tickets than children in the joyful or neutral conditions," reports the Association for Psychological Science blog. 

b) Take a moment to classify the two variables in Study 1:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

c) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

d) And now for Study 2:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

e) Which kind of experiment is Study 2: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

Look back at the description of the study. Did either Study 1 or Study 2 manipulate "taking kids to an art museum"? Nope. In fact, Study 1 took place online (kids watched video clips on their own computers). And Study 2 took place in a science museum. Neither of the studies manipulated art museum exposure. Art museum tickets were used in Study 2, but only as a way of operationalizing generosity.

That factual error didn't stop the journalist from asserting:

Parents ... can always go to their local art museum to give their kids a guaranteed dose of awe. 

Guaranteed dose of awe? That conclusion does not really follow from this study!  So here are a couple more questions. 

f) What could be an alternative headline for the Inc.com story--one that is more accurate to what the study actually manipulated?

g) The journalist appears to have generalized from one setting (watching video clips) to another (going to an art museum). Which of the four big validities is the journalist treading on here? 

h) Here's a construct validity question: The two studies operationalized the dependent variable, "generosity" in two different ways. Were each of them a good way to measure this variable? Why or why not? What are the strengths of operationalizing the variable more than one way?

i) Now for a question related to internal validity: The main independent variable was the content of the film clip the children saw. To ensure internal validity, what should the researchers hold constant about the film clips used in the three conditions? 

 

 

 

Rebound relationships

Starting a relationship soon after a breakup is associated with some positive outcomes for people. Photo: SynthEx/Shutterstock

Did you know that psychologists have studied rebound relationships? I didn't, at least not until I encountered this summary of some rebound relationship research in Forbes. The title of the journalist's piece reads, "A psychologist explains 3 ways a rebound relationship can benefit your mental health."  

It may interest you that this research goes against conventional wisdom. In this case, the conventional wisdom is that rebound relationships are bad for you, perhaps because they mean one is not adequately processing or mourning one's old relationship. However, you should be skeptical that the causal claim in the headline is actually supported by the studies. 

We'll talk about two of the studies here. The first one suggests a rebound relationship might help "acceptance of your breakup". The summary reads:

Contrary to popular belief, the mental health of people who opt for rebound relationships isn’t really worse off than people who choose to remain single after a significant breakup. In fact, according to research published in the Journal of Social and Personal Relationships, people who went for new relationships shortly after ending one were more resolute about their decision to break up with their ex-partner.

The "people who" language that is used here seems to hint that we're dealing with a correlational study, not an experimental one. And that makes sense, right? It's unlikely we'd be able to manipulate the "rebound relationship" variable by randomly assigning some recently broken-up people to have a new sweetheart or not. 

Let's walk through why this study is correlational.

a) In the text above, the first measured variable is contained in this phrase: "people who went for new relationships shortly after ending one". The variable could be named "how fast you entered a new relationship". What might this variable's levels be?  Why is this variable probably measured?

b) the second measured variable is mentioned here: "were more resolute about their decision to break up with their ex-partner." The variable could be named "Degree of confidence in the break-up decision". What might this variable's levels be? Why is this variable probably measured?

c) Now sketch a scatterplot!  Take your answers to a) and b) and put them on the x and y axes of a scatterplot. Then add the points. Will your dots slope positively? Or negatively? 

d) Now think about causation. What might be the temporal sequence of these two variables? The journalist implies one direction (A --> B):  that the quicker breakup came first, and caused people to become more resolute about their decision. But what about B --> A. Could that causal direction plausibly work here? 

e) Another aspect of causation is internal validity, or third variable problems. Sometimes a third variable, C, might go with both A and B, and be responsible for their correlation. For example, maybe people whose previous relationships were shorter were more likely to date new people sooner, and people in shorter previous relationships were also more resolute about their decisions to break up. Can you think of another C variable that might be responsible? Explain how your C variable goes with both A (how fast they entered a new relationship) and B (degree of confidence in break-up decision). 

The third study concerns how a rebound relationship might provide  "a boost to your self-esteem". Here's the journalist's summary:

[After a breakup,] you may feel unattractive, lonely, and just generally low ...

Psychologist Claudia Brumbaugh told BBC in an interview that, according to her research, rebound relationships can help people feel “more confident, desirable, (and) loveable. Possibly because they had proven it to themselves. They had more feelings of personal growth and independence. They were more over their ex, they felt more secure. There were no cases where people who were single were better off.”

f) In this study, the levels of the first measured variable are contained in the phrases: "rebound relationships" and "people who were single". What might you name this variable? Think of a name that captures both levels at once.  Why is this variable probably measured?

g) The second measured variable is mentioned here: "more confident, desirable, (and) loveable.." In fact, this probably represents three variables. What might be the levels of each of these variables?  Why are they probably measured?

h) Now sketch a bar graph.  Look back at f) and g) (picking just one of the variables in g for the y-axis). Then estimate the heights of the two bars. Why did we suggest a bar graph here, and not a scatterplot? 

i) Now think about causation. What might be the temporal sequence of these two variables? The journalist implies one direction (A --> B):  that the rebound relationship (or p0st-relationship singlehood) came first, and caused people to become more confident, desirable, and loveaable. But what about B --> A. Could that causal direction plausibly work here? 

j) Another aspect of causation is third variables. Sometimes a third variable, C, might go with both A and B, and be responsible for their correlation. For example, maybe people who are more extroverted more likely to start new relationships, and maybe people who are extroverted were also more likely to feel confident and desirable. Can you think of another C variable that might be responsible? Explain how your C variable goes with both A (being in a rebound relationship vs. being single) and B (more confidence or desirability)?

 

Multivitamins and memory in older adults

Participants in the study took either a multivitamin or a placebo.
Photo: Sorapop Udomsri/Shutterstock

When you shop in a natural foods or general nutrition store, you might see claims that vitamins can improve your health and cognitive function. Scientists don't always test such claims in randomized trials, so it's best to be skeptical until you read research. But a recent study did test whether taking a multivitamin can improve people's memory. In this blog post, we'll check out how an NPR journalist summarized the study.

Before reading on, think about this: How would you design a study to test this question? What would be your independent and dependent variables? What kind of study would you use--specifically, what type of experiment (Posttest only? Pretest/posttest? Concurrent measures? Repeated measures?)

Here's the journalist's summary of the study:

A team of researchers wanted to assess how a daily multivitamin may influence cognitive aging and memory. They tracked about 3,500 older adults who were enrolled in a randomized controlled trial. One group of participants took a placebo, and another group took a Silver Centrum multivitamin, for three years. The participants also took tests, administered online, to evaluate memory.

At the end of the first year, people taking a multivitamin showed improvements in the ability to recall words. Participants were given lists of words — some related, some not — and asked to remember as many as possible. (List-learning tests assess a person's ability to store and retrieve information.)

People taking the multivitamin were able to recall about a quarter more words, which translates into remembering just a few more words, compared to the placebo group.

a) Make a table of the variables in this study:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) This was an experiment. How do you know--what are some key words used in the journalist's description?

c) Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

d) Sketch a graph of the study's results. Here's a hint: You should put time (or pretest/posttest) on the x-axis). 

Next we'll apply the four big validities to this study. 

Construct validity
e) Ask a question about the construct validity of the dependent variable.  What do you need to know in order to assess construct validity of the DV?

Statistical validity
f) One of the questions to ask is about effect size: How much better was the memory in the multivitamin group?  The NPR report offered two quotes related to effect size.  One is from the study's own authors: "'We estimate that the effect of the multivitamin intervention improved memory performance above placebo by the equivalent of 3.1 years of age-related memory change,' the authors write in their paper"

The other is from an independent researcher, who "says the overall effect found in the study was quite small. 'It seems like a pretty modest difference'" 

Do these two people disagree about the study's effect size? 

g) Another aspect of statistical validity is precision. The sample size of the study is relevant here--why? What effect would a large sample (of over 3,000 participants) have on the confidence interval of the estimate?

h) A third aspect of statistical validity is replication, or what else is known? Here's a relevant quote: "Study author Dr. JoAnn Manson...says this is not the first study to show benefits from multivitamins. She points to a study published last year in Alzheimer's & Dementia that showed participants who took a daily multivitamin performed better, overall, on global cognitive function on tests measuring story recall, verbal fluency, digit ordering, as well as executive function."

External validity
i) What do you need to know to evaluate whether the results of this study can generalize to, say, older adults in North America? (check your answer--you did not talk about sample size, did you?)

Internal validity
j) This study is described as a "double-blind, randomized, placebo-controlled" design. Knowing this, you can rule out many internal validity threats. Briefly explain why the study allows you to rule out:

• Selection effects
• Placebo effects
• Testing threat
• Maturation
• History
• Observer bias

At the end of the article, NPR reported

The study was funded, in part, by the National Institutes of Health and other grants. The vitamins were provided by Pfizer Inc. and Haleon, the makers of Centrum, the brand of multivitamins taken by participants in the study. The study authors say the funders had "no role" in the study design, analysis or interpretation.

Why is this statement important for observer bias? 

 

 

 

 

Does cannabis make people more creative?

To measure creativity in the study, participants came up with creative uses for a brick. Photo: levkro/Deposit photos

A lot of people believe that using marijuana can make them more creative--some artists use it specifically when they are composing songs, painting, or creating new work. 

A recent study experimentally tested this idea. The study provides a useful example of a null effect. Here is a summary by a journalist in the Washington Post. 

The journalist writes:

...researchers designed a randomized controlled trial comparing the creative outputs of light cannabis users who had just imbibed vs. those who had not.

In a first set of experiments, about 107 light users of cannabis were asked to take a standard creativity test within 15 minutes of getting high. For the control group, the researchers asked 84 other participants to do the task only if they had not used cannabis in the past 12 hours, by when any effects would have worn off.

For the creativity test, the participants were asked to consider a brick, and generate as many creative uses as possible for it in four minutes. As expected, participants who were under the influence felt more jovial, which made them feel like their ideas were more creative than sober participants.

But creativity assessments by third-party raters, who did not know who was under the influence, were a buzzkill. When they rated the responses on novelty and usefulness, they did not see a difference in creativity.

Questions

a) Start by making a table of the variables in this study.

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) Check your work above. There were two operationalizations of the dependent variable, creativity. Do you have both of them?  Do you also have the dependent variable of joviality? 

c) Was this study an experiment or a correlational study? How do you know?

d) Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

e) Sketch two graphs of the study's results: One graph for each dependent variable. (Hint: The x-axis will show the same independent variable in both graphs). 

f) The results which showed that, when creativity was measured by third-party raters, there was no effect of using cannabis on creativity. This is a "null effect" as introduced in Chapter 11. Flip to Table 11.3 in your text: we'll apply several of the questions to this study. 

• Could there have been an ineffective manipulation of the IV? 
  
  
• Is there evidence of a ceiling or floor effect on the IV or DV?
  (Hint: the fact that there were differences in joviality and self-reported creativity suggest that the IV actually did work--right?)
  
  
• What are some potential causes of within-groups variability in this study? How might the researchers have minimized their effects?  Consider situation noise, measurement error, and individual differences. 

 

Do puppies remember their siblings?

The studies found that puppies recognize their mothers and that mothers recognize their offspring, but that siblings may not recognize each other after they've lived apart. Photo credit: cynoclub/Deposit Photos

Not an endorsement, but our dog Fig likes to use Rover.com, and the company sent me this article in their email newsletter. 

It's titled, "Do puppies remember their siblings?" I'm happy to talk about any kind of dog research, so let's dive in!

The article covered a couple of studies that tracked how long puppies would remember their siblings and mothers. Before you read on, consider: How could you measure whether puppies remember these early relationships? You can't ask them, so what behaviors might help you decide?

Here's the Rover description of one of the studies.

One study used scent-based tests to determine whether dogs recognized their mother and siblings two years after leaving the litter by the age of 12 weeks. They found that by the age of 2, dogs could only recognize a sibling they lived with—not any of their other siblings.

But dogs could recognize their mothers after two years of separation, and the mothers also recognized their puppies.

The snippet above doesn't provide a lot of information about how the scent-based tests were conducted. However, by looking at the original empirical article, we can learn more details. Specifically, the researchers placed blankets in the beds of dogs' littermates, mothers, and other unrelated dogs that were matched on sex and age. Then they conducted three studies.

In the first one, they tested mother dogs. The mother dogs were presented at the same time with a cloth from one of their offspring (who they'd not seen for about 2 years) and a cloth from an unrelated dog of the same sex and age. Fourteen out of 18 of the mother dogs spent more time sniffing the cloth from their own offspring.

In the second study, offspring were tested. The researchers presented offspring with a blanket from their own mother and from another female dog of the same age. Nineteen out of 25 offspring spent more time sniffing the cloth from their own mother. 

In the third study, dogs were presented with a blanket from one of their own siblings, whom they had not been with in over 2 years, and a blanket from another dog of the same age and sex. Among the 25 dogs who lived alone, they showed no preference for one blanket over another. Among the 23 who dogs lived with one of their siblings (they'd been adopted together), most showed a preference for the blanket from one of their siblings (not the one they lived with) compared to the control blanket. 

a) Start with the first study (with mothers). Make a table of the variables in this study:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

b) Also with the first study: Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

c) Now do the with the third study (with siblings). Make a table of the variables in this study:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

d) Also with the third study: Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

Sketch a graph of the results of the third study. Remember to put the DV on the y-axis. 

Now for another study from this article:

Another, smaller study also supports the importance of scent in recognizing dog and human family members. Eight dogs separated from their mother for varying lengths of time—from several months to several years—still reacted more strongly to her scent than the scent of another female dog.

And here's a study that tested whether dogs could recognize the smell of the humans who raised them as puppies (but whom they no longer lived with):

In the same study, researchers also considered nine dogs reared by humans from birth who were separated from their owners for varying lengths of time—from 11 months to just over three years. After this time, the dogs spent longer reacting to their owner’s scent than the scent of a same-sex human who lived with a dog of the same breed and gender.

e) In the "recognize the humans" study, classify the variables: 

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

f) Also with the first study: Which kind of experiment is this? Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

g) You might have noticed that all of these studies used the same design (did you get concurrent measures?). Why? Could you design an independent-groups version of one of these studies? What might be the advantages and disadvantages of that choice? 

Gendered racism (intersectionality) and psychological distress

The study found a correlation between gendered racism and symptoms of depression among Black college women. Photo: Deposit Photos

This post is about a study on intersectionality--a combination of race and gender--in the lives of Black women in college. The empirical study appeared in the Journal of American College Health, and a journalist from PsyPost summarized it on the PsyPost website.

To start, here's the journalist's introduction to gendered racism, which is related to intersectionality:

Racism and sexism are both forms of discrimination that can take a significant psychological toll on the person suffering from it. Black women in particular are a group who often suffer from the combination of both forms of oppression, which can be referred to as gendered racism.

And here is an overview of the study's methods: 

For their new study, the researchers utilized 164 Black, female participants who were undergraduate or graduate students at a primarily White public university in the Southeast United States to serve as their sample. Data was self-reported online during the fall of 2016. Participants completed measures on demographic information, gendered racial microaggressions (e.g. “Someone accused me of being angry when I was speaking in a calm manner”), depression, and psychological distress.

One of the results reported by the journalist is the following: 

There was ... a relationship between frequency of gendered racism microaggressions and depression, with those experiencing more instances of microaggressions experiencing increased depression. 

a) Using the table below, identify the variables in this study:

Variable name	How was this variable operationalized? (if you can't really tell, say so)	Is this manipulated or measured?
		(add rows as needed)

 

b) Is this a correlational or experimental study? How do you know?

 

c) Sketch a scatterplot of the study's results (as described above).  Label your axes mindfully.

 

The PsyPost article included these two caveats: 

This study took important steps toward understanding negative outcomes associated with gendered racism. Despite this, there are limitations to note. One such limitation is that due to the cross-sectional nature of this study, causation cannot be inferred. Additionally, data was only collected from one, primarily White institution and may not be generalizable to other schools.

d) Let's dig into the first caveat, "One such limitation is that due to the cross-sectional nature of this study, causation cannot be inferred." Apply the three criteria for causation here.
i) First, does the study have covariance?  
ii) Second, why does the study not have temporal precedence?   
iii) Third, what is a third variable (a "C" variable) that might independently go with both gendered racism microaggressions AND with higher levels of depression? 

e) Now let's address the second caveat, "data was only collected from one, primarily White institution (PWI) and may not be generalizable to other schools".

Which big validity is this caveat addressing? 

f) Do you think that the association they found between gendered racism microaggressions and depression might be stronger, weaker, or the same if they conducted the study at a historically Black college or university (HBCU)?  Explain your reasoning. 

(Note: In question f, you are thinking about a moderator! Specifically, you are thinking about whether the type of institution (PWI vs. HBCU) moderates the relationship between gendered racism microaggressions and depression).

g) How could you interrogate the construct validity of this study? List two questions you could ask (one for each major variable).