Everyday Research Methods

Our ingroup members aren't stinky (A factorial design)

How bad would these sweaty jerseys smell to you? Probably depends on how much you like the team. Photo by Jim Rogash/Getty Images

A Pacific Standard journalist ran a nice description of a factorial design, subtitled "new research finds body odor is less disgusting if we share an identity with the smelly person in question." The study they were reviewing was a factorial design. Read about it and try to figure out the study's IVs and DVs.

The introduction to the study focuses on tribalism and group identity. Group identities can includeour family memberships, teams we belong to, or our regular social groups--even the universities we attend. The Pacific Standard story reports:

"Group identities affect not only social perceptions, but also our basic sensual experiences," writes a research team led by Stephen Reicher of the University of St. Andrews and John Drury of the University of Sussex. Their paper is published in the Proceedings of the National Academy of Sciences.

In this study, the researchers approached the issue from the opposite perspective, asking if an item that is physically disgusting (a sweaty T-shirt) is less repulsive if it was soiled by "one of our own."

The disgusting T-shirts were created by asking a research assistant wear each shirt for a week, even during workouts and while sleeping.

After making a few shirts smelly in this way, the researchers conducted two studies. Here is a description of one of the studies they reported. It was a factorial design, so as you read it, think about what its IVs are. 

[The study] featured 90 students from St. Andrews University who sniffed a smelly T-shirt emblazoned with either the logo of their school, that of a nearby rival university, or no logo at all. ...their instruction sheet stated that researchers were measuring the ability of either "St. Andrews University students" or, simply, "students."

Here are the results:

Participants whose identity as a St. Andrews student was made salient "went to wash their hands more quickly, and used more soap," after smelling a shirt with either their rival's logo or no logo.

They rushed to the hand sanitizing station after smelling the shirt with the rival school's logo, getting there in three and one-half seconds (on average). In contrast, those who smelled a shirt with their own school's logo ambled over; it took about six seconds. It seems they were significantly less bothered by the stink.

[The results showed that] Thinking of yourself as a "St. Andrews student" reduced disgust only for fellow St. Andrews students, but conceiving your identity in broader terms—as "a student"—reduced disgust for all students.

Questions about the study:

a) What is one of the independent variables? How many levels does it have, and what were the levels?

b) What is the other independent variable?  How many levels does it have, and what were they?

c) What was the dependent variable?  (There seem to be a couple of dependent variables but the results focus on one of them).

d) State the design (e.g. 2x2? 3x3?  Is it independent groups? Within groups? mixed?)

e) The description allows you to sketch a graph of the results. Give that a try! Remember to put the DV on the y axis, one IV on the x-axis, and another IV as the color of the bars or the lines.

f) Now take a look at your graph, and think about whether there are main effects and interactions in this pattern. Is there a main effect for the first IV? If so, describe it ("such that...").  Is there a main effect for the second IV? If so, describe it. Is there an interaction? If so, describe it.

Selected answers

a) - c) This is a 2x3 independent groups factorial design with "identity" as the first IV (2 levels: "St. Andrews" or "A student") and "shirt" as the second IV (3 levels: St. Andrews, Rival school, or no logo). The dependent variables were how fast people walked to the sanitizing station and also how much soap they used. 

e) You can see the actual graph of the results by clicking here. The authors put the 2-level variable on the x-axis, but you could also graph it with the t-shirt variable on the x-axis. 

f) Here's how the authors of the original described the results for the DV of how fast people walked: 

Time spent walking to the hand sanitizer.

There was a significant main effect of t-shirt [St Andrews, M = 6.06, SD = 1.76; Dundee, M = 4.52, SD = 1.50; plain, M = 4.09, SD = 0.69, F(2, 79) = 17.12, P < 0.001,η² = 0.30]. Post hoc Tukey tests revealed that participants smelling a St Andrews University t-shirt took significantly longer to walk to the hand sanitizer than participants who smelled a Dundee University or plain t-shirt (both P < 0.001). There was no significant difference in time walking between the Dundee University and plain t-shirt conditions. This main effect was qualified by a significant interaction between t-shirt and identity salience: F(2, 79) = 4.22, P = 0.02, η² = 0.10 (Fig. 3).

 

"Parents of difficult children use ipads to pacify kids"

Parents reported giving children ipads frequently when the children are upset. Does the study match the headline provided by the journalist? Photo: Nadezhda1906/Shutterstock

The headline for a story on Tech Times reads, "Parents of 'difficult children' likely to use ipads to pacify kids." The story is about a survey reported in the medical journal JAMA Pediatrics. 

According to a new study by researchers from the C.S. Mott Children's Hospital at the University of Michigan, parents from low-income families were more likely to give mobile devices to calm children with social and emotional difficulties.Published in the journal JAMA Pediatrics, the researchers recruited 144 healthy children between the ages of 15 and 36 months from low-income families. The researchers asked the parents how often they give or allow the use of mobile devices during a variety of situations. Different scenarios include eating, being in public, during chores, when the children are in distress and at bedtime.

What were the results? 

They found that parents were more likely to use mobile devices as a coping strategy to pacify children who are having tantrums to keep the peace at home.

a) What kind of claim is it to say "Parents of 'difficult children' likely to use ipads to pacify kids" (Frequency? Association? or Causal?)  
What are the two variables in this claim? 

b)  Consider each of the two variables in this claim.  How did they operationalize each variable in the study? Was each variable measured or manipulated? 

c) Is this a correlational study or an experiment?  How do you know? 

d) The headline implies that the parents used ipads mainly  "to pacify difficult children". Is that accurate, according to what you've read about the study's methods? Did they actually ask about children's levels of behavioral problems? 

e) Sketch a carefully-labelled graph of the result that they described, especially in light of your answer to d) above. 

 

Suggested answers

a) This is an association claim, because it's only stating that one variable goes with another variable. The two variables suggested in the claim are 1) Using mobile devices or not and 2) Degree of difficulty of the child's behavior.

b) According to the journalist's coverage, these two variables were operationalized via self-report, apparently through a questionnaire. Both variables were apparently measured (it states that the researchers "asked the parents..."). I think the first variable could be called "using a mobile device or not" and the second one could be the situation (In public, doing chores, in distress, at bedtime). 

c) This seems to be a correlational study because the key variables were measured via self-report. 

d) Notably, the method seems to suggest that parents were asked how often they use ipads in different situations, and the results indicated that parents reported using the ipads more often at home during tantrums. Therefore, the results and the method that were described don't seem to measure "difficult children" but rather "difficult situations." 

e) To graph this outcome: They found that parents were more likely to use mobile devices as a coping strategy to pacify children who are having tantrums to keep the peace at home, you could have created a bar graph, with different situations on the x-axis (such as "having tantrums," "bedtime," "during chores" etc.) and have the level of using the ipad on the y-axis. The bar for "having tantrums" would presumably be the highest bar. 

Alternatively, you could make a scatterplot, with "difficulty of the situation " on one axis, and "likelihood of using ipad" on the other axis, and the scatterplot would have a positive slope.

 

Psychology's reproducibility: The conversation continues

In case you missed it, there have been some updates to last year's news about the replicability of psychological science. In this space last year, I blogged about the reproducibility project, which had reported that fewer than half (47%) of the psychology studies they attempted to replicate actually replicated. You can read my post for students here and you can read about the original study here. 

Very recently, an independent set of researchers (Gilbert, King, Pettigrew, and Wilson) published an open letter in the journal Science, outlining three major problems they had with the reproducibility project, and arguing that the project underestimated the true reproducibility of psychological science. Their view is more optimistic. You can read that open letter here.

A couple of days after their letter came out one of the authors of the original reproducibility project (Nosek) submitted a rebuttal to at least one of the critiquesYou can read that here.

Then a day after that, the Gilbert team addressed the Nosek team's argument. You can read that one here. 

Are you keeping track?  Suffice it to say that the debate on how to estimate the reproducibility of psychological science studies is lively and dynamic!  If you teach research methods to undergraduates, you might find the discussion difficult to summarize for students with little experience with the statistical issues raised in this discussion. What do you think? Have you got any ideas about how to discuss these issues with your students? Email me at [email protected].