This example gives you practice extending a simple experiment (Chapter 10) into a factorial design (Chapter 12).
Think back to the last time you asked a question in your research methods class--or any other class. What was the professor's reaction to your question? Maybe they said, "That's a great question!" or maybe they said, "Didn't we just go over this?" As you probably know, it matters how professors respond--you might feel validated, encouraged, rejected, or ignored, all of these, depending on the professor's tone.
A group of researchers studied how a professor's responses can make students feel, especially in STEM (science, technology, engineering, and math) courses. They used a scenario method in which students imagined asking a question in class. The researchers summarized their research for a general audience on the SPSP news website. Here's why they were interested in this topic:
Students' confidence and belonging may influence their interest and motivation in STEM. Confident people believe that they can succeed. Belonging refers to how much people feel they fit in and are accepted. We expected that when students imagined their instructors responding to their questions in a positive way, students would feel more confident and accepted, which would boost their motivation and interest in STEM.
Study 1 (Chapter 10)
The researchers conducted five studies. Let's talk about Study 1:
College students imagined receiving a response from an instructor to a question they asked. The response was either positive ("That's a great question, I'm glad you brought that up"), negative ( "I'm not sure why you're asking this question, we went over this already") or neutral ("We're actually out of time today, so please hold your question till next time"). They then reported their feelings of confidence, belonging, interest in joining the research lab, and intentions to recruit others to the lab.
When students imagined receiving a positive rather than a negative or neutral response from their STEM instructor after asking a question, they felt more confidence and belonging in STEM.
a) Classify the variables in this study: (Hint: There's one IV and four DVs)
Variable name | What are the variable's possible levels? |
Is this manipulated or measured? | Is this an IV or a DV? | For IVs: is it within groups or independent groups? |
b) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? How do you know?
c) Sketch a bar graph or line graph of the results they described (select one of the four DVs).
Study 2 (Chapter 12)
In another study, the researchers compared whether the feedback's effects depended on the type of student. Here's the theoretical reasoning:
Some students may be more vulnerable to these effects because they doubt their belonging or ability in certain areas. For example, being 'bad at math' is a common stereotype that negatively affects women's experiences in STEM. Women often feel uncertain about their belonging in STEM and perform worse when reminded of negative gender stereotypes about women's math abilities. Based on these ideas, we reasoned that women, compared to men, would react more strongly to imagining receiving certain types of instructor responses to their questions in STEM contexts.
Here's the study's results:
Indeed, compared to men, women reported greater confidence, belonging, and intentions to join a STEM lab after imagining their instructor responding to their question with, "That's a great question, I'm glad you brought that up." Women also reported lower confidence, belonging, and intentions to join a STEM lab after imagining a negative instructor response or even a neutral instructor response of, "We're actually out of time today, so please hold your question till next time."
d) Classify the variables in this study: (Hint: The IV and three DVs are the same as before, but now there is a participant variable (PV)/independent variable :
Variable name | What are the variable's possible levels? |
Is this manipulated or measured? | Is this an IV, PV or a DV? | For IVs and PVs: is it within groups or independent groups? |
e) Which kind of experiment is Study 2: It's a factorial design, so state it like this ___ X ___ and state whether it's an independent groups factorial, within groups factorial, or mixed factorial design.
f) Sketch a bar graph or line graph of the results they described (select the confidence DV). I recommend putting the "feedback condition" variable on the X axis and the Gender PV as a line or bar color.
g) Can you detect an interaction in the data? How about a main effect for type of feedback?