Everyday Research Methods

In U.S. football, Whiter-looking men more likely to be head coaches

Football coaching seems to have a stereotype problem
Photo credit: dotshock/Shutterstock

Here's a story from the website Psypost. The headline reads, "Study of U.S. football coaches finds the more White you look, the more likely you are to be head coach".

This report is a study based on a well-known disparity in U.S. football (both college and pro): Even though 57% of college players are Black, Whites are more likely to be in football coach positions. Among college teams, for example, position coaches are around 31% Black; only 9% of head coaches are Black. The higher you look in the coaching leadership hierarchy, the more likely you are to see White men (no women are head coaches of college teams). 

A research team got together to document this disparity and also to study whether men who looked especially "White" were even more likely to be head coaches. In their study, the research team obtained coaches' photos and titles from team publicity programs; all coaches were in the college Football Bowl Subdivision (FBS). Over 100 Black and White coaches were in the sample. In addition to recording the coaches' roles (such as position coach, defensive or offensive coordinator, or head coach) the team also coded how "stereotypically White" or "stereotypically Black" each coach appeared. "Stereotypicality" was coded by showing the photographs to a new sample of Americans, who were asked to rate each coach on a 5-point scale according to how stereotypically White (or, if they were Black, how stereotypically Black) they looked. 

Here's one of the researchers, Melissa J. Williams, explaining to the Psypost journalist some of the background for the study:

“Race and ethnicity have both internal aspects (such as the importance a person places on their cultural heritage) and external aspects (such as how other people perceive a person’s physical appearance). Each of these aspects has a different effect on the role that race plays in a person’s life.  In terms of the specific setting we explored in this paper, football leadership represents an interesting collision of historical stereotypes about Black and White individuals. Black individuals, especially Black men, are stereotyped as good at sports and as physically imposing – characteristics that go with being good at football. Whereas White individuals, especially White men, are stereotyped as confident and charismatic – characteristics that go with being good at leadership. That combination of stereotypes got us interested in looking at who makes it to the top in football coaching, and how their appearance plays into their success.”

Here's how the journalist summarized the study's results:

The findings revealed that White coaches were more likely than Black coaches to occupy leadership roles. Moreover, among White coaches, the more stereotypically White their appearance, the greater the likelihood they occupied a leadership role. When it came to specific roles, appearing stereotypically White increased the likelihood of being a head coach among White coaches. By contrast, appearing stereotypically Black decreased the likelihood of being a head coach among Black coaches.

a) This was a correlational study. What were the measured variables? (There are three variables mentioned in the paragraph above).

b) This result presents a moderator. One way to draw it is to create two matching scatterplots: One for White coaches and one for Black coaches. What would go on the x and y axes of the two scatterplots? What would the slope of each scatterplot be?

c) Write a moderator sentence of this form: _________ moderates the relationship between ________ and __________, such that ________________.

After documenting the correlation between appearing White and being a head coach, the team conducted a second study, an experiment: 

In a second study, Williams and team had a sample of 375 football fans view photos of real college football players. Through a series of trials, participants were shown side-by-side images of two football coaches [both White or both Black], with one being high and the other low in racial stereotypicality. Participants were asked to select which coach they felt would be promoted to head coach in the future.

The results revealed that when the two players were White, participants chose the more stereotypical photo as more likely to become head coach in 70% of trials. But when the two players were Black, participants chose the more stereotypical photo only 52% of the time.Thus, stereotypicality had a stronger effect on being perceived as a leader for White compared to Black football players.

c) What are the independent variables in this experiment? What is the dependent variable?

d) To study the "stereotypicality" variable, this second study used a concurrent measures design. Can you explain why?

e) The first study was correlational and the second was experimental. Why is it helpful to have one of each study type in this investigation? 

At the end of the Psypost article, the researchers were quoted about their study: 

Overall, the researchers said their findings, “provide insight into the insidious nature of racial stereotypes, the role of physical appearance in perpetuating them, and how leadership attainment may be constrained for those who do not look the part.”

Here's a link to the empirical journal article, which includes photos of coaches who were coded as higher or lower on the stereotypicality scale: Personality and Social Psychology Bulletin. 

 

 

 

 

 

 

Time alone with your thoughts?

Credit: Ground Picture/Shutterstock

Imagine sitting alone with your thoughts for 20 minutes. You wouldn't be allowed to distract yourself with a book, phone, or even walking around. How much do you think you would you enjoy it?  

When this question was asked of participants in a series of studies, most reported that they'd not enjoy time alone very much.  But this study also tested how well their predictions matched their actual experience. The empirical research was published in the Journal of Experimental Psychology: General, and  summarized in a trade magazine for the American Psychological Association. According to the journalist, 

...study lead author Aya Hatano, PhD, of Kyoto University in Japan[, said] “Our research suggests that individuals have difficulty appreciating just how engaging thinking can be. That could explain why people prefer keeping themselves busy with devices and other distractions, rather than taking a moment for reflection and imagination in daily life.”

The research team conducted six studies. Here is an overview of some of them: 

In a series of six experiments with a total of 259 participants, the researchers compared people’s predictions of how much they would enjoy simply sitting and thinking with their actual experience of doing so. In the first experiment, they asked people to predict how much they would enjoy sitting alone with their thoughts for 20 minutes, without being allowed to do anything distracting such as reading, walking around or looking at a smartphone. Afterward, participants reported how much they had enjoyed it.

The researchers found that people enjoyed spending time with their thoughts significantly more than they had predicted. This held true across variations of the experiment in which participants sat in a bare conference room or in a small, dark tented area with no visual stimulation; variations in which the thinking period lasted for three minutes or for 20 minutes; and one variation in which the researchers asked people to report on their enjoyment midway through the task instead of after it was over. In every case, participants enjoyed thinking more than they had expected to.

a) The researchers conducted six different studies. Do these sound like direct replications? Replication-plus-extensions? or Conceptual replications? (you can skip this question if you haven't done Chapter 14 yet)

b) The journalist identifies these as "experiments," so let's assume that each study contains a manipulated variable. Take this study, as described by the journalist:

The researchers found that people enjoyed spending time with their thoughts significantly more than they had predicted [in a variation of] the experiment in which participants sat in a bare conference room or in a small, dark tented area with no visual stimulation. 

In this study, one of the independent variables was "predicted vs. actual".  What is the other main independent variable, or IV? What is the dependent variable, or DV? 

c) Do the same for this study:

variations in which the thinking period lasted for three minutes or for 20 minutes;

In this study, one of the independent variables was "predicted vs. actual". What is the other IV? What is the DV?

d) And for this one: 

In another experiment, the researchers compared one group of participants’ predictions of how much they would enjoy thinking with another group’s predictions of how much they would enjoy checking the news on the internet. Again, the researchers found that people underestimated their enjoyment of thinking. The thinking group expected to enjoy the task significantly less than the news-checking group, but afterward, the two groups reported similar enjoyment levels.

In this study, one of the independent variables was "predicted vs. actual". What is the other IV? What is the DV?

e) Challenge question: For the study above (comparing time alone to looking at news on the internet)  and graph the result. You can compare your graph to Figure 6 in the empirical article (available next to the APA journalist's story, here).

Here's a final caveat about the results. I appreciated this statement because it puts the statistical outcome in concrete and realistic terms: 

It is important to note that participants did not rate thinking as an extremely enjoyable task, but simply as more enjoyable than they thought it would be, according to Murayama. On average, participants’ enjoyment level was around 3 to 4 on a 7-point scale. Future research should delve into which types of thinking are most enjoyable and motivating, according to Murayama. “Not all thinking is intrinsically rewarding, and in fact some people are prone to vicious cycles of negative thinking,” he said.

 

 

Diet + Exercise: A factorial example

The study found that both exercise and diet are important. Is the effect additive or interactive? Photo credit:Dragana Gordic/Shutterstock

You might have heard someone claim that because they exercise regularly, they can eat whatever they want and still be healthy. When people say this, they seem to mean that either exercise OR diet can be sufficient to foster a healthy lifestyle. However, a recent study counters this "either/or" view. Like many other studies, this one found evidence that both diet and exercise contribute to health outcomes in adults.

In this blog post, we'll explore how the study can be viewed as a factorial design (Chapter 12), and we will discuss how its main finding can map on to different factorial outcomes. 

The empirical study appeared in the British Journal of Sports Medicine, which may be paywalled. I learned about the study from number of journalists who covered it, including this accessible summary in Healthline, which we'll use in this post.  

Here are the opening bullet points from the Healthline story: 

• A new study finds that in order to live longer, people need to eat healthily and exercise regularly.
• While a balanced diet and regular exercise independently have plenty of health benefits, the study notes the greatest reduction in mortality risk comes from combining diet and exercise together.

a) These bullet points mention the study's three main variables. One of the variables is mortality risk, or longevity (lifespan). What are the other two variables?

b) Chapter 12 concerns factorial designs, in which there are two (or more) independent variables (IVs) and one dependent variable (DV). Reflect back on the three variables you listed in a), above, and identify which are the IVs and which is the DV.  In addition, identify the levels of each IV. 

Here are some details about the study's method and results, as summarized by the journalist:

For the study, the researchers evaluated the health and exercise data of 346,627 individuals over the course of 11 years. They tracked how much exercise each participant completed in an average week along with how intense their physical activity was. The team also followed the participants’ eating habits. During the study window, 13,869 participants died — 2,650 from heart disease and 4,522 from adiposity-related cancers.

The research team found that any type of regular exercise was associated with a lower risk of mortality, and those who both ate well and exercised had the lowest risk of dying.

In addition, while exercise and diet are independently linked to a lower risk of mortality, high levels of exercise cannot entirely mitigate the harms of a poor diet.

I made up three patterns of data to illustrate possible outcomes of this study. You can see the graphs below, labelled Outcomes A, B, and C. Take a moment to orient yourself to the graphs, noticing how the DV is on the y-axis, one IV is on the x-axis, and the other IV is depicted using line color:

 

c) Which of these outcomes (A, B, or C, above) is consistent with the idea that EITHER exercise OR diet alone is sufficient to maintain a longer lifespan? Explain your reasoning.

d) The journalist wrote that "The research team found that any type of regular exercise was associated with a lower risk of mortality, and those who both ate well and exercised had the lowest risk of dying [i.e., the longest lifespan].”  Two of these outcomes (A, B, or C) are consistent with this statement. Which ones? 

e) In your own words, describe the difference between Outcome A and Outcome B.

f) Based on what the journalist wrote, can you tell which of these two (Outcome A or Outcome B) the study found? Specifically, which one could go with how the journalist described the study's outcome:

“In our study, those who ate a poor-quality diet and were active still had substantially reduced mortality risk than those who ate a poor-quality diet and were inactive,” Melody Ding, MPH, PhD, the lead author of the study and an associate professor of health and medicine in the Sydney School of Public Health at the University of Sydney, told Healthline. “It’s just that you had the maximum risk reduction when doing both things right.”

g) Just for more practice: Two of the outcomes above depict an interaction effect (an interaction between diet and exercise). Which 2 outcomes are they? Explain your reasoning--how do you know?

Suggested answers:

a) Besides longevity, a second variable is diet quality and a third variable is exercise habits.

b) Longevity is the DV. Diet quality is an IV (levels could be good vs. poor diet). Exercise habits are also an IV (levels good be regular exercise vs. no exercise)..

c) Outcome C. Outcome C shows that you have the same lifespan if you exercise or eat a good diet, and that doing both does not increase your lifespan any more than doing one or the other (all three conditions--exercise alone, diet alone, and both--show the same lifespan). Note that this is not actually the result that the study found.

d) Both Outcome A and Outcome B are consistent with the journalist's statement. 

e) The difference between Outcomes A and B can be summarized with the terms "additive" versus "interactive."
Outcome A shows an additive effect--A good diet adds (in this example) 3 years to your life; exercising adds 3 years, and if you do both, it adds 6 years. One is good, both is better.  
Outcome B shows an interactive, or multiplicative effect. It shows that if you do both diet AND exercise, you have an especially long lifespan. That is, in this example, a good diet adds 3 years to your life; exercising adds 3 years, and if you do both, it adds 11 years. One is good, both is amazing (kaboom!). In Chapter 12 you'll learn that this is an interaction--the combination of the two IVs results in something even bigger than simply adding the two together.

f) In my view, we can't tell which outcome the study found based on what the journalist wrote. However, since they used the terms "both" rather than "especially", it seems like it might be Outcome A. My reading of the empirical study backs this up--it seems to be an additive effect (not interactive). 

g) Outcome B and C both show interactions, but of different types. First, you can see the interactions in both because the lines are not parallel--there are differences of differences. That is, the effect of diet is different, depending on if we're talking about regular exercise or no exercise. 

In Outcome B, there is an interactive, or multiplicative effect--I sometimes call this an "especially" interaction. It shows that if you do both diet AND exercise, you have an especially long lifespan (compared to doing either one alone).

In Outcome C, there is a different kind of interaction--I might call it an "only if" or an "it depends" interaction. You could explain it by saying that lifespan is reduced only if you don't exercise AND don't eat well. If you exercise, or diet, or both, you'll live longer. 

That baby smell...

The study found an interaction effect: Gender of participant interacted with the smell condition they were in. Credit: HayDmitriy/Deposit Photos

If you’ve ever smelled a baby’s head (I know....it can be irresistible!), you probably inhaled a variety of chemicals, including one called HEX.

A group of researchers recently published a study in which they exposed men and women to this particular chemical. They suggested that, given human evolutionary theory and emerging research on how different smells affect us, the smell of HEX might impact people's behavior. In the study, they focused on the behavior of aggression, using a laboratory game. The research was summarized by a journalist for the Voice of America.

The researchers’ hypotheses were derived from human evolutionary theory related to aggression:

…lead author Eva Mishor, who was studying signals of aggression for her doctorate at the Weizmann Institute, noted that in animals, female aggression is usually aimed at defending their young, while male aggression is often directed at the offspring themselves.

Specifically, the authors demonstrated that in a childrearing context, smelling HEX (the baby-head chemical) led to different responses in males than females.

The study tested people's responses to HEX using rigged games designed to aggravate the player. In one game, when the aggravated player is allowed to win, he or she gets to blast the opponent with a loud noise. The louder the noise, the higher the scientists rated the player's aggression level.

[Among players who] sniffed HEX before playing [compared to players who had sniffed a control chemical], women's blasts were louder and men's were quieter.

In case you didn't notice, this study used a 2x2 factorial design with an IV and a PV. Let’s walk through it.

a. What is the first factor (the independent variable?) What are its 2 levels? Was it manipulated as independent-groups or within groups?

b. What is the second factor (the participant variable?) What are its 2 levels? Was this variable independent groups or within groups?

c. What was the dependent variable?

d. Read the description of the results in the last quoted paragraph above. Then, sketch a little graph of the results they are describing (please assume that, in the non-HEX control condition, men are a bit more aggressive than women).

e. The results of the study show an interaction. Describe the interaction in your own words. 

f. The journalist describes the results like so:

The effect was somewhat subtle. On a six-point scale, the differences were, on average, roughly between half a point and less than a point in either direction.

Here the journalist seems to be characterizing the effect size, and seems to be referring to the 95%CI of the difference. This is unique; most journalists do not mention the size of such effects or their CI.

Suggested answers:

a. The first factor was exposure to HEX (or not). This variable was manipulated as a between-groups variable—people played the aggression-inducing game either after smelling HEX OR after smelling a control (non-HEX) scent (i.e., there were different people in each condition). 

b. The second factor was the participant variable of participant gender. It’s a participant variable because there are two levels (male and female) that cannot be truly manipulated. This is an independent-groups variable because people were either male or female.

c. The dependent variable was level of aggression, as operationalized as noise blasts.

d. Here's a sketch I made, based only on the journalist's description. You should look at the actual results in Figure 2 of the open-access article in Science Advances.  Because the researchers also coded noise blast intensity over time, their results figure includes that factor as well. 

 

 

 

Does mindfulness make people selfish? It depends....

Mindfulness instructions appeared to interact with people's independence/interdependence in this set of factorial studies.  Photo: Rob Wilkinson/Alamy Stock Photo

A new set of studies on mindfulness and selfishness provides an opportunity to practice skills related to factorial designs (Chapter 12). Let's get started.

 

Here's the theoretical background of the study, as described by a BBC journalist. He explains that mindfulness can 

sharpen our concentration, extend our working memory and boost our mental flexibility. With practice, we should become less emotionally reactive – allowing us to deal with our problems more calmly. 

One ‘benefit’ that you might not expect to gain, however, is heightened egotism. Yet a recent study suggests that, in some contexts, practicing mindfulness really can exaggerate some people’s selfish tendencies. With their increased inward focus, they seem to forget about others, and are less willing to help those in need. 

The journalist then describes two studies run by psychologist Michael Poulin. The first compared the effects of a mindfulness meditation session among people from two different cultures. Notice the term "depends" in this description, which is a signal that the study was investigating an interaction. The pair of studies comes....

from Michael Poulin, an associate professor in psychology at the State University of New York at Buffalo, who wanted to investigate whether the effects of mindfulness might depend on its cultural context and the existing values of the people who are practicing it.

He was particularly interested in the ways people think about themselves – their “self-construal”. Some people take a more independent viewpoint, focused on personal characteristics. If they are asked to describe themselves, they might emphasise their intelligence or their sense of humour. People with an interdependent view, on the other hand, tend to think of themselves in terms of their relations to others. 

Read this description of the study, and identify the first independent variable (IV, or factor), the second  IV (factor), and the dependent variable:

... Poulin invited 366 college students into the lab and first gave them a questionnaire measuring their independence or interdependence. 

Half were then asked to perform a meditation focused on the sensation of breathing. The control group were given a “sham” meditation that involved sitting and letting their mind wander for 15 minutes. The exercise may have been relaxing, but it wasn’t designed to increase their mindfulness. 

Next came a test of pro-social behaviour, in which the students were told about a new project to help fund a charity for the homeless. They were then given the opportunity to stuff envelopes ...which would be sent to the university’s alumni – but they were told there was no obligation to do so, if they wished to leave early.

Now for the results:

Sure enough, Poulin found that the effects of the meditation depended on people’s existing attitudes. If they were already interdependent, then the people who took the mindfulness exercise were willing to spend much more time on the charitable task; overall, they stuffed about 17% more envelopes than the control group. If they were independent-minded, however, the exact opposite occurred – the mindfulness had made them even more self-centered, so they were less willing to help the homeless. Overall, they stuffed around 15% fewer envelopes than the control group. 

Questions about Study 1:

a) What was the study's first factor (Hint: It was an IV)? Was it independent groups or within groups?

b) What was the second factor? (hint: it was a participant variable, or PV). Was it independent groups or within groups?

c) What was the DV?

d) State the design of this study using this template: "__x__ , [independent-groups/within-groups/mixed] factorial"

e) Sketch a bar or line graph of the results, as described in the paragraph above. use Figures 12.16 and 12.17 as models.  Do you see an interaction here? 

Now here is a second study, as described by the same journalist:

To be sure the finding was robust, Poulin’s team conducted a second experiment, in which the participants were first given a short text written either in the first-person singular (I), or first-person plural (we). As they read the text, they had to click on all the pronouns – a simple task known to prime either independent or interdependent thinking. They then completed the meditation tasks and, to test their pro-sociality, were asked whether they wanted to devote time to chat online with potential donors for the homelessness charity. 

Once again, the mindfulness exercise exaggerated the effects of their self-perception, driving increased altruism among the interdependent-minded, and decreased altruism among the more independent-minded. 

Questions about Study 2:

f) What was Study 2's first factor (Hint: It was an IV)? Was this one independent groups or within groups?

g) What was the second factor? (This was also an IV). Was it independent groups or within groups?

h) What was the DV in this study?

i) State the design of this study using this template: "__x__ , [independent-groups/within-groups/mixed] factorial"

j) Sketch a bar or line graph of the results, as described in the paragraph above. use Figure 12.11 as a model. Do you see an interaction? (You should, because this study was described with the phrase, "it depends"!)

k) There's causal language in the journalist's summary (e.g., " the mindfulness exercise ...was driving increased altruism" and " the mindfulness had made them even more self-centered"  Is causal language warranted here? How come? 

Note:Poulin's study has been peer reviewed and will be appearing in the journal Psychological Science, in the future. You can read an early version of the paper here. 

 

Saturated fat can impair concentration

Breakfast of (concentration) champions? Maybe not....Photo credit: Kyndall Ramirez/Unsplash

A headline at Psych Central reads, "High fat meal may impair concentration." Does the study behind the headline support that causal claim?

At first, you might assume that this came from a correlational study on diet and concentration. However, it turns out the research summarized in the article was experimental: It manipulated the saturated fat content of one specific meal. In the study, researchers created two versions of a breakfast--one made with saturated fat and the other made with sunflower oil, which is also high in dietary fat but is not the saturated type. 

The journalist describes the method of the study this way: 

Women in the study [n = 52] completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

The high-fat meal followed: eggs, biscuits, turkey sausage and gravy containing 60 grams of fat, either a palmitic acid-based oil high in saturated fat or the lower-saturated-fat sunflower oil. Both meals totaled 930 calories and were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Five hours later, the women took the continuous performance test again.

Between one and four weeks later, they repeated these steps, eating the opposite meal of what they had eaten on the first visit.

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment.

(The original empirical journal article is here; it might be paywalled)

Questions:

a) What was the independent variable (IV) in this study and what were its levels?

b) What was the dependent variable?

c) Was the IV manipulated as independent groups or within-groups?

d) This design is difficult to classify because it has a pretest and posttest, but the women also experienced both levels of the IV. Therefore it shares features with two of the four designs in Chapter 10. What are those two designs it shares features with? (Note to students: As you can see in this example, the four designs you learned about in Chapter 10 are not the only types of experiments there are, and this study provides an example of alternative designs.) 

e) Sketch the results of this study--using "reduction in concentration" (i.e., a difference score) as the dependent variable. 

Next let's work through the four big validities. 

f) Here's some text related to external validity of this study:

For this work, Madison [the lead author] conducted a secondary analysis of data from Kiecolt-Glaser’s study assessing whether high-fat meals increased fatigue and inflammation among cancer survivors.

and

[the experimental meals]  were designed to mimic the contents of various fast-food meals such as a Burger King double whopper with cheese or a McDonald’s Big Mac and medium fries.

Given the information above, to what kinds of people and what kinds of situations might this research be generalized? How do you know? 

g) The researchers found an 11 percent drop in concentration after the meal high in saturated-fat. What further information would you like to know in order to evaluate the study's statistical conclusions? 

h) Can this study support the causal claim that "A high-fat meal may impair concentration"? Apply the three criteria to the study: Covariance (what did the results show?), temporal precedence (did the method ensure that the meal came before the drop in concentration?), and internal validity (What potential confounds did they control for in the study?) 

i) Here's some information about the study's construct validity: 

Women in the study completed a baseline assessment of their attention during a morning visit to the lab. The tool, called a continuous performance test, is a measure of sustained attention, concentration and reaction time based on 10 minutes of computer-based activities.

What questions would you need to ask to determine if this test is a good measure of people's ability to concentrate? 

Instructors: If you teach Chapter 12, you can also use this example as a mixed factorial design, because the study also included a participant variable of endotoxemia, a condition called "leaky gut." From the journalist's summary: 

After eating the meal high in saturated fat, all of the participating women were, on average, 11 percent less able to detect target stimuli in the attention assessment. Concentration lapses were also apparent in the women with signs of leaky gut: Their response times were more erratic and they were less able to sustain their attention during the 10-minute test.

“If the women had high levels of endotoxemia, it also wiped out the between-meal differences. They were performing poorly no matter what type of fat they ate,” Madison said.

Students could identify the IV, PV, and DV and practice graphing the results described above. 

Thanks to my colleague Stephen Chew, who shared this example.

Research on Native American mascots

The College of William and Mary has phased out its Indian-themed mascot over the years. In 2010, the College replaced its former mascot with "Griffin", above. Photo: Mitchell Layton/Getty Images

Since 2005, the NCAA has required colleges and universities to reflect upon, and if necessary, change sports team mascots that feature depictions of Native-Americans.  Now a team of psychology researchers have published an investigation about support for such mascots among different groups of people. The set of studies  includes both correlational and experimental designs, including one testing whether the mascots can affect people's donations to the universities. 

The lead researcher, Dr. Michael Kraus, told a journalist what led to his interest in the research question. 

“I was at the University of Illinois at the time, and despite their Native American mascot being officially discontinued in 2007, it was surprising to see the mascot on campus in 2013. The continued presence said something about the norms on campus, and what kinds of students feel safe and welcome there,” Kraus said.

The journalist summarized two studies conducted by Kraus and his team. Here's the first:

In a survey of 201 students, the researchers found that those with lower levels of explicit prejudice toward Native Americans reported lower levels of belonging at the University of Illinois.

In other words, students who disagreed with statements such as “It is now unnecessary for the U.S. government to honor their treaty obligations to Native tribes” also tended to disagree with statements such as “I identify with the university” and “I show school spirit regularly.”

In addition, students with higher levels of explicit prejudice, as well as white students, were more likely to have positive views of the retired “Chief” mascot.

Questions

a) This is a bivariate correlational study. What are its two main variables?

b) The journalist gives some clues about how each of these two variables was measured. What kinds of measurements were used--self-report? physiological? observational? What do you think of the items? 

c) Sketch a scatterplot of the results that are described. Label your axes carefully.

d) Can this study support the causal claim? Why or why not? (Apply the three causal criteria; when you come to internal validity, specify at least one third variable that might be associated with both measured variables.)

The empirical article contains several studies on this topic. One of them tested the impact of  the University of Illinois' retired mascot, Fighting Illini, on donations to the school. 

The experiments showed that stereotypic Native American imagery “reduced actual donations to the institution by 5.5%, and was particularly likely to decrease (increase) donations among those low (high) in prejudice,” the researchers wrote.

In the study, researchers had online participants read about different universities, one of which was the University of Illinois. Some participants  were randomly assigned to see the Native American mascot, Chief Illini, next to the school name, while some did not.  Then participants had the chance to distribute $2 in donations to the four universities. When people had been exposed to the  Native American mascot, donations to the University of Illinois decreased. 

But like all research, the study includes some caveats. “Probably the biggest one is that we really only studied these dynamics at the University of Illinois, but we expect these norms to operate similarly everywhere. Someone, maybe us, needs to do that additional work,” Kraus said.

More Questions

e) Is the donation study an experiment or a correlational study? What are its two main variables?

f) When Kraus states that "we expect these norms to operate similarly everywhere," to which of the four big validities is he referring?

You can access the empirical research here.

Instructors: The complete package of studies is a complete set with a frequency claim (via an observational study of mascot use on campus) , a correlational study, and an experimental study.

 

 

 

 

Interested students might learn from Dr. Stephanie Fryberg's work, showing how American Indian high school students respond to being reminded of professional sports mascots that feature American Indians.

Does this punctuation make me sound angry, or what.

Which of these two text messages seems the most sincere? Photo: Travis Carr

If you're an American young adult, you probably already follow certain unwritten rules about about punctuating text messages. FOR EXAMPLE, EMAILS AND TEXTS WRITTEN IN ALL CAPS ARE CONSIDERED A FORM OF SHOUTING. Text messages follow their own set of norms, and the punctuation we use can lead to different interpretations of the same words. Some linguistics researchers have studied the impact of different text-message punctuation. 

This brief article in Quartz summarizes the results of some of this research. The journalist introduced the research with this: 

Though periods can still signal the end of a sentence in a text message, many users will omit them (especially if the message is only one sentence long). This tendency now subtly influences how we interpret them.

First example

This study is an example of a 2x2 factorial design. Here's how the journalist summarized the study: 

In 2016, psychologist Danielle Gunraj tested how people perceived one-sentence text messages that used a period at the end of the sentence. Participants thought these text messages were more insincere than those that didn’t have a period. But when the researchers then tested the same messages in handwritten notes, they found that the use of a period didn’t influence how the messages were perceived.

a) What are the two independent variables (IVs) in this design? What are their levels?

b) What was the dependent variable? 

c) Do you think the IVs were manipulated as independent groups or within groups? 

d) Sketch a graph of the results of the study. Remember to put the DV on the y-axis. Put one IV on the x-axis and represent the other IV with two different lines or bar colors. 

 

Second example

For practice with a simple experiment (not a factorial), compare these two text messages. Which one do you think participants found to be more sincere?

JACKIE I AM SO SO SO SORRY! I thought you were behind us in the cab and then I saw you weren’t!!!!! I feel soooooooo bad! Catch another cab and ill pay for it for youuuuu

Jackie, I am so sorry. I thought you were behind us in the cab and then I saw you weren’t. I feel so bad! Catch another cab and I’ll pay for it for you.

e) These two messages were studied by linguist Deborah Tannen. What would have been the independent and dependent variables in the study that compared these two messages? How might you design the study--would a posttest only design be better, or a concurrent-measures design, and why? 

f) How might the results of this study change if they were conducted on an older sample compared to a younger one? 

Tilt your head down to appear more intimidating

According to the study, Captain Marvel's eyebrows have something to do with her dominant pose. Credit: Collection Christophel/Alamy Stock Photo

Captain Marvel looks tough. Perhaps one reason is the tilt of her head, which enhances the appearance of dominance, according to a series of studies that appears in Psychological Science. The studies were summarized by journalist Alice Walton for Forbes. 

Here are some excerpts from the journalist's description of the study:

...there’s a way to communicate dominance that doesn’t involve moving the facial muscles at all: just tilt your head downward slightly and maintain eye contact.

In one experiment, participants in the University of British Columbia study rated [several] avatars who were tilted upward 10 degrees, tilted downward 10 degrees, or level (neutral). Then they rated statements designed to gauge how dominant the person appeared, including “This person would enjoy having control over others,” and “This person would often try to get his way regardless of what people may want.”

Participants rated the downward-tilting heads significantly more dominant than either of the other two positions. When the experiment was repeated using photos of actual people rather than avatars, the same pattern was found.

a) The journalist labels this an experiment, which it really is. What is the independent variable? What are its levels? Is the IV manipulated as independent groups or within-groups? (By the way, the Forbes article shows photos of the three levels.)

b) What is the dependent variable in this study?

c) What's the experimental design: Posttest only? Pretest-posttest? Repeated measures? Concurrent measures? 

d) Sketch a bar graph of the results described here: "Participants rated the downward-tilting heads significantly more dominant than either of the other two positions."

e) What do you think of the construct validity of the dependent variable? Does this operationalization really get at "dominance?" Why or why not? What questions from Chapter 5 might you ask here? 

f) The journalist mentions that "the experiment was repeated using photos of actual people rather than avatars". It's an example of replication. Which type of replication study would you call it: direct replication? conceptual replication, or replication plus extension? (Chapter 14 defines these terms.)

g) By replicating the avatar study on actual people, which validity (or validities)  are these researchers enhancing? 

More Advanced Questions

After establishing the basic effect, the researchers conducted some additional studies to test the mechanism, or mediator. Read this paragraph (written by the journalist) to decide what the mechanism was, and think about how they tested it. 

In another experiment, the team found that the key to the effect was the eyes and eyebrows: when they showed participants faces with the eyes/eyebrows occluded, the tilted head/dominance effect disappeared. But then they showed participants pictures of just the eyes and brow portion of the face, the effect was present, which suggests that this part of the face is both necessary and sufficient—meaning it's not only required but is all that's necessary to elicit the effect.

The above argument suggests that the researchers' theory goes like this:

    Tilting head downward ---> changes in eyebrow position --> dominant appearance

h) To find evidence for their theory, the authors tested four conditions in a 2x2 factorial design. Downward tilt (vs. neutral), and face only vs. eyes/eyebrows only. You can see examples of the stimuli they used in Figure 4 of the original article in Psychological Science. Sketch the predictions of these two studies. Like this:

If changes in eyebrow position are important for tilted faces to appear dominant, then the results should show [sketch a result].

If changes in eyebrow position are NOT important for tilted faces to appear dominant, then the results should show [sketch a result].

 

Did the show "Thirteen Reasons Why" cause an increase in suicides?

The study used a quasi-experimental design.  Credit: Beth Dubber/Netflix/Kobal/Shutterstock

Update 7/19/2019: Netflix has decided to remove the scene depicting the main character's actual suicide, perhaps in response to data from this and other studies. Here's one story on the change. 

The Netflix series "Thirteen reasons why" tells the story of a teenage girl who commits suicide, leaving behind audio recordings about people who contributed to her decision. When the series was released in April, 2017, mental health and school professionals became concerned about the possibility that young viewers of the show would experience increased risk of suicide themselves. The phenomenon of "suicide contagion" occurs when a publicized suicide leads to other suicide attempts. Young people may be particularly susceptible to its effects. Suicide is the third leading cause of death among American teenagers (ages 12 to 19, according to the CDC). 

This Mashable journalist covers an empirical journal article published in JAMA Psychiatry in 2019.  The Mashable story reports: 

A new study, conducted by different authors, came to a similar conclusion: In the three months after 13 Reasons Why launched, the youth suicide rate for 10- to 19-year-olds rose unexpectedly by 13 percent. Based on historical trends dating back to 1999, 94 more American adolescents and teens died by suicide during the three-month period than otherwise expected. Similarly, the study published last month looked at suicide deaths in the nine months following the show's launch and found an additional 195 fatalities. 

Students and instructors will find it useful to consult the publicly-available JAMA Psychiatry article that contains graphs of this study. Specifically, the solid orange lines in Figure 2 show the suicide rates for 24 months of time: 15 months before the show debuted, and about 9 months after. 

a) This study used a quasi-experimental design. Which of the designs from Chapter 13 was it? (Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design?)

b) In the quote above, the journalist reported that an 13 percent increase in suicide equals an additional 94 adolescent teen deaths. This figure captures the effect size of the study's result. What do you think of this effect size? Would you call it small or large? 

The empirical journal article also documents the publicity the show received via social media. Here's how the journalist describes the evidence: 

Niederkrotenthaler and his co-authors looked at potential exposure to the show between April 1 to June 30, 2017, when the first season of 13 Reasons Why had just debuted and was most popular on Twitter and Instagram. Using social media data as a proxy for understanding peak interest in the show, the researchers suggest the show's portrayal of suicide created a 'contagion effect' The phenomenon can happen when media coverage of suicide leads to suicide clusters. 

Students and instructors can consult the empirical article's Figure 1 to see the results of the social media analysis. 

b) Why did the authors of the JAMA article include the social media piece? How does this evidence help them make their argument? 

The researchers cannot establish for certain that viewing the show caused the increase in suicide. Here's the Mashable journalist: 

While the researchers couldn't determine whether the children who took their own lives watched the series, an accompanying editorial written by researchers at Harvard and Stanford argues the study provides "strong evidence" that the Netflix show may have led to that increase. 

"One cannot draw definitive causal conclusions from such data, but that no similar increase was seen in suicide rates other than for the age group to which the media portrayal pertained provides some further compelling evidence that the excess suicides may indeed have been owing to the series," the editorial's authors wrote. 

c) The quote above adds an additional detail about the study. The researchers not only tracked the suicide rates of people aged 10-19; they also tracked suicide rates of people in the 20-29 and 30+ age group. The researchers found an increase in suicide after April 2017 only for the teenage group. How does this pattern of results support the argument that the show led to increased teen suicide? 

If you want to talk to someone or are experiencing suicidal thoughts, text the Crisis Text Line at 741-741 or call the National Suicide Prevention Lifeline at 1-800-273-8255. Here is a list of international resources.