Everyday Research Methods

Some sports psychology behind the Eagles' Super Bowl win

 

How could you test the hypothesis that the most successful pro athletes are more creative than average?
Photo: Timothy A. Clary/AFP via Getty Images

The Eagles are the most local team to my university (the University of Delaware is less than one hour from Philly), so we were excited when they won the Super Bowl last month! (Notice the BIRGing I just did?)

A Drexel University-based sports psychologist, Eric Zillmer, wrote this column in The Conversation that applies some sports psychology findings to the Eagles' win. It's a great introduction to sports psychology, and how it can help us understand the game at a deeper level.  (Instructors: The questions I am asking here are producer-oriented, but you can adapt the examples according to the goals of your own course.)

The first section of Zillmer's article refers to the personality profiles of successful, high-profile athletes. The most research-valid way to assess personality is with the Big Five, which consists of extroversion, conscientiousness, agreeableness, neuroticism, and openness to experience. Zillmer mentions the following results:

Research suggests that super-elite athletes are tough-minded and not easily rattled.

Their psychological profiles look similar to those of high-performance solo classical guitarists or fighter pilots. On personality tests, athletes typically score at least average in extroversion, openness and agreeableness, and high in conscientiousness. Professional athletes work incredibly hard and are disciplined, well organized, goal-oriented, reliable and generally sociable.

A new focus in personality research in competitive athletes is on creativity and, specifically, being open to experience, which includes being receptive to new ideas and being flexible.

Openness to experience and a related trait, creativity, are important to sports performance:

Creativity is essential in unscripted football plays – when a planned play has not been executed properly, like a fumble or an interception. [...]. it becomes even more critical during improvised offensive plays when everything is unscripted. In a sport where milliseconds matter, being creative and engaging in something your opponent doesn’t expect can be the difference between winning and losing.

a) Think about how you could design a study to test the hypothesis that openness to experience is associated with being a more successful professional athlete. Imagine that you have a large pool of athletes of varying ability who are willing to answer questionnaires for you. What traits would you measure in this sample? What variables do you predict would be associated with that trait, and how would they be associated? (sketch a scatterplot, with the trait on one axis, and the athletic outcome variable on the other axis).

 

The second section of the article has to do with the role of anxiety in sports performance:

Performance anxiety is a leading cause of poor athletic performance. Research suggests an athlete’s competitive anxiety can be cumulative and maybe even be contagious, affecting teammates negatively.

Since emotions and thoughts affect behavior and performance, the concept of emotional self-regulation – or intentionally focusing on the present moment – has been introduced into competitive sports. Mindfulness, meditation, yoga, breathing exercises and grounding techniques are now integral to the toolkit for high-performance sports.

b) Think about how you could design a study to test the hypothesis that mindfulness training can improve the performance of professional athletes. Because this is a causal hypothesis, you'd conduct an experiment--what would the independent variable be? What would the dependent variable be?  Would it make more sense to use an independent-groups design or a within-groups design? 

c) Alternatively, you could read about this study, which tested mindfulness training on the anxiety of competitive shooters. How could you adapt this study to a sample of professional football players? 

The third section of Zillmer's article has to do with fan behavior. He writes:

Fans... [are often] are nervous wrecks, like parents watching their kids compete. One remedy for managing this stress is watching the game with other fans. 

d) The study that Zillmer linked to in his article didn't actually measure fans' anxiety. So, how could you design an experimental study to test the hypothesis that watching a big game with other fans can help reduce the fans' anxiety abut their team? What would the independent variable be? What would the dependent variable be? 

 

 

Guess what's contagious in chimpanzees?

Given the study's results, predict which of these chimps is most likely to pee after another one does.
Photo: Dirk M. de Boer/Shutterstock

The radio program Short Wave (on NPR) highlights 3 science-related news stories every week. In this episode, one story is about a study showing that peeing is contagious in chimpanzees. The study is an excellent example of observational research, statistical reasoning, and turning an informal observation into a full-fledged empirical study.

You can listen to the story here--the study is explained in the first 3 minutes of an 8-minute broadcast. I'm going to paste sections of the transcript here, which features a conversation between reporters Regina Barber, Jonathan Lambert, and Ari Shapiro:

BARBER: [...] Ena Onishi, a grad student at Kyoto University in Japan, was studying a group of captive chimpanzees when something similar struck her as odd.

ENA ONISHI: I noticed that they seemed to have a tendency to urinate around the same time. And it kind of reminded me of some human behaviors of going to the bathroom together, for example.

[...]

LAMBERT: Yeah. So this observation got Ena wondering if this behavior might be socially contagious, like yawning. And to see if it was, she spent more than 600 hours watching a group of 20 chimpanzees.

[...]

LAMBERT: [...] she said that it was easier to hear them pee than to see them pee. But so she noted, when each individual chimp peed and where they were relative to each other. And looking at the data altogether, an interesting pattern emerged.

BARBER: So basically, chimps were peeing together a bit more often than you'd expect if they were just peeing at random. And she published that conclusion in the journal Current Biology this week.

SHAPIRO: Could she tell why this was happening?

BARBER: So being closer helped, but proximity wasn't, like, the main factor here. Chimps have a hierarchical society, and it turns out that the lower ranking chimps were more likely to catch the urge to pee from more dominant chimps.

SHAPIRO: So when my boss pees, I'm going to pee?

BARBER: Yeah.

SHAPIRO: Is that what you're telling me?

LAMBERT: Maybe. And they don't know why this is. It could just be that lower ranking chimps are paying closer attention to higher ranking ones. But that's just one possibility.

SHAPIRO: OK, so there's clearly more to study here, but is there any evolutionary reason that contagious urination would be an advantage?

LAMBERT: One idea is that doing the same thing together just kind of helps a group sync up, which could help them operate better as a unit. If this happens in the wild, it might help the chimps avoid predators who get attracted by the smell of pee by concentrating it all in one spot.

BARBER: But we can't rule out that there might be, like, a non-adaptive reason. It could be just that the chimps pee when they hear other chimps pee, sort of like humans get the urge when you hear, like, running water.

In the empirical study, Onishi and her co-authors counted up (for a total of 600 hours) how often an instance of one chimpanzee's pee co-occured within 60 seconds of another one's pee. This was an instance of "pee contagion." Then, they compared their observed number of c0-occuring pees to a simulation model. In the simulation model, a computer estimated the total number of co-occuring pees they'd see in 600 hours if the null hypothesis is true--that is, how often would the pees occur within 60 seconds of each other if there's no contagion? They found that the observed data showed more pee contagion than the simulated model, confirming that there was contagious peeing.

Then they correlated "pee contagion" to two predictors. First, they found that contagion was more likely when two chimpanzees were physically closer to each other: There was more contagion when chimps were arms' reach, compared to 1 meter or more than 3 meters away. Second, they correlated pee contagion with each chimp's position in the hierarchy, and found a positive correlation, such that the higher the first chimp was in the hierarchy, the more likely another chimp would contagious pee after they did. 

Questions

a) Reading the transcript (or listening to the story), what was Onishi's informal observation, and how did she turn it into a study? Reflect on how her research started with an informal hunch, and evolved into a full-fledged published article. Do you have any research hunches that you'd like to turn into a published article?

b)  The commentators bring up three very different explanations--mechanisms--for the contagious peeing effect. Identify the three explanations in the story.

c) Pick one of these three explanations. How could you collect data in a new study to test that explanation? What variables would you measure, and what result would support the explanation? 

The Current Biology article is available here. You can see scatterplots of pee-contagion and dominance, and dot plots of pee-contagion by distance away. (may be paywalled)

When questionable survey methods reach the Supreme Court

According to the report, the Court may have been misled by the quality and true findings of survey on the habits of gun owners. 
Photo: B Christopher/Alamy Stock Photo

Polling research can play a critical role in shaping public opinion and public policy, so it's crucial that it be conducted well. Polling (survey) research is increasingly used in court cases, too. Because survey research can have an important impact, it's important to scrutinize a survey's research methods.

An example from the United States is a landmark gun-rights case (known as the Bruen case) from 2022, one of many pro-gun rulings in recent years.  In the Bruen case, the Court ruled that New York State could not outlaw concealed-carry handguns. In their written opinions, several of the justices cited polling research that supposedly found that  guns are "commonly" used for self-defense. 

The full story can be found here, and it was covered in the podcast, The Daily, on June 20, 2024 (available here).

According to reporting by New York Times journalists, the U.S. Supreme Court's decision in the Bruen case depended, in part, on the results of expert research: a survey of gun owners conducted by Georgetown business professor William English. English's survey data was used to argue that many gun owners use guns for self-defense. The Court accepted the research as expert testimony and allowed it to influence their reasoning.  However, when journalists started looking into the survey and the researcher who conducted it, they found instances of questionable scientific practice.

First, the researcher who conducted the survey has not made himself available to journalists to answer questions about his research and he also did not respond to requests that he testify in court about his work on the survey. This reticence to answer questions about one's research goes against scientific norms of transparency, or as Merton called it, communality. As the textbook says, "Scientists should transparently and freely share the results of their work with other scientists and the public", p. 15. From the journalist:

Government lawyers tried to talk with [Dr.English] in December, after his research surfaced in a Firearms Policy Coalition lawsuit seeking to overturn Washington State’s ban on assault weapons. But Dr. English did not respond to emails, phone calls or a certified letter, and a process server bearing a subpoena could not find him at the university or his home, where someone inside “looked at the door, then turned away,” according to court records.

Faced with efforts to compel his testimony, the pro-gun plaintiffs in [a different] Washington case chose to drop all references to his research.

Second, the Dr. English misled people about his impartiality as a scientist. Specifially, the journalists learned that the surveys he conducted were funded by pro-gun legal groups connected to the National Rifle Association. Failing to disclose one's funding is a potential conflict of interest and violates Merton's norm of disinterestedness, which says "scientists strive to discover the truth, whatever it is; they are not swayed by conviction, idealism, politics, or profit," (p. 15). From the story:

Dr. English seems at first glance to be an impartial researcher interested in data-driven insights. He has said his “scholarly arc” focuses on good public policy, and his lack of apparent ties to the gun lobby has lent credibility to his work.

But Dr. English’s interest in firearms is more than academic: He has received tens of thousands of dollars as a paid expert for gun rights advocates, and his survey work, which he says was part of a book project, originated as research for a National Rifle Association-backed lawsuit....

At least one of Dr. English's papers on the gun survey in question (which is published here) does not disclose this funding. 

Third, the journalists reported that Dr English's research methods probably led to misleading results and interpretations of the data. Specifically, the journalists reported:

... the wording of some questions could elicit answers overstating defensive gun use, and that he cherry-picked pro-gun responses.

An example comes from the self-defense question on the survey. In reporting his results, English asserted that 31% of his survey respondents said "yes" when asked, "Have you ever defended yourself or your property with a firearm, even if it was not fired or displayed?" But in that presentation, English left out the full text of the question. The Times report highlighted in blue the omitted portions:

Many policymakers recognize that a large number of people participate in shooting sports but question how often guns are used for self-defense. Have you ever defended yourself or your property with a firearm, even if it was not fired or displayed? Please do not include military service, police work, or work as a security guard.

In Chapter 6, you can read that this is a potential example of a leading question. Gun owners might be more motivated to answer "yes" to it, given its lead-in.

Here's another example of leading question wording (with a leading portion highlighted in blue), coupled with English's misleading presentation of results:

Some have argued that few gun owners actually want or use guns that are commonly classified as ‘assault weapons.’ Have you ever owned an AR-15 or similarly styled rifle? You can include any rifles of this style that have been modified or moved to be compliant with local law. Answering this will help us establish how popular these types of firearms are.

Elsewhere in his findings, Dr. English quotes about 30 gun owners attesting to the usefulness of large-capacity magazines for self-defense — but only two reported actually firing a gun, both at animals. He did not quote any of the hundreds of negative responses from gun owners, including: “There is no reason to have a magazine like that unless you’re fighting in a war.”

 

The journalist concludes that while English's research was only one piece of the full argument in Bruen and other pro-gun rulings, "...his credentials and findings carried weight. At least five briefs cited his research."

Reflection Questions:

a) What's the difference between writing a leading survey question and writing a misleading summary of one's research? 

b) What do you think might have happened in the court cases if Dr. English had been more up-front about who funded his research? 

c) What do you think might have happened in the court cases if briefs about Dr. English's presentation of his research had included the full texts of the questions he used?  

 

 

What's your love language?

Are there really only five love languages? Does everyone have only one primary love language? Do couples who match--have the same love language--experience better relationships? Photo: Olesia Bilkei/Shutterstock

It's nearly Valentine's Day, so it's a good time to feature this new story on so-called "love languages." According to a new review article by relationships researchers, it looks like we should move love languages into the category of "things that make research psychologists cringe" (along with the Myers Briggs Type Inventory, learning styles, and others).

Many journalists have covered an empirical review article recently published in the journal, Current Directions in Psychological Science. Here's an example of coverage from website Medical Express:

Even if you don't know your love language, you've probably heard of the concept. The theory's pervasiveness in pop culture has only increased in the 30-odd years since Baptist pastor Gary Chapman published his book "The Five Love Languages: The Secret to Love That Lasts." [...]

"We were very skeptical about the love languages idea, so we decided to review the existing studies on it," says Emily Impett, a professor in the UTM department of psychology who collaborated with UTM graduate student Gideon Park and York University Assistant Professor Amy Muise. "None of the 10 studies supported Chapman's claims."

Chapman's original book argues that there are five love languages: physical touch, words of affirmation, acts of service, quality time and gifts. He also argues that each person has a primary preference for one of these. Furthermore, he argues in his book that when two partners have the same love language (for example, both of them prefer physical touch) that their relationship will be better.

The journalist explains that Impett and her team addressed three weaknesses in Chapman's theory. The first element they investigated was whether people only have one primary love language. While Chapman's q"uiz" does identify one primary love language for each respondent, in fact, because of the way it is set up, Chapman's test forces people to compare each type to the others, so people must choose one. In contrast, Impett and her colleagues wrote that when people are provided a Likert scale and asked to rate each love language individually, most people report multiple preferences of showing and expressing love.

A second critique of Chapman's original thinking was that it was based on a restricted sample (mostly White, religious, middle-class Americans). In other cultural contexts, there may be different ways of expressing love. 

But let's focus on their third , and most important, claim:

Most importantly, Impett and her team found no scientific evidence for Chapman's central contention that people who choose partners that speak their love language, or learn to speak it, will have more successful relationships.

The researchers reviewed the literature and found

...no scientific evidence for Chapman's central contention that people who choose partners that speak their love language, or learn to speak it, will have more successful relationships. "There's no support for this matching effect," says Impett. "People are basically happier in relationships when they receive any of these expressions of love." 

This conclusion is a great example of the theory-data cycle, so let's walk through it. I've included Figure 1 for your reference.

a) The theory-data cycle starts with a theory. What is Chapman's theory about love languages? Remember that a theory is a set of general statements about how variables are related to each other, and Chapman's theory had several ideas. What are his main theoretical ideas?

The next step in the cycle is to select a particular research question. Let's work with the research question about matching love languages. The question might be, "do couples with matching love languages have better relationships?"

Then we design a study to test the research question. In one study that Impett and her team reviewed, researchers recruited a sample of couples and measured each person's love language. They divided the couples into two types: those who had the same love language and those that did not. They measured each couple's relationship satisfaction.

b) The next step in the theory-data cycle is to state a hypothesis for the study--what should the results of this specific study be if the theory is true? State specifically what the results should be--you can even sketch a graph of the predicted outcome.

c) The next step is to evaluate the data in light of the theory. What did the study actually find? Do these results support the theory, or not? That is, which arrow in the Figure above (the right one or the left one) will be be applying in this case?

In addition to providing data to counter the "love language" theory, Impett and her colleagues proposed an alternative metaphor: a "balanced diet." Specifically, they reported that most people appreciate many kinds of ways of showing love--not just one--and that we can all benefit from receiving some of each type, no matter what type it is. 

Challenge question:

d) Come up with a study that might test the "balanced diet" view of love expressions. What variables would you measure? How should the variables be related to each other if this theory is true?

 

 

 

What's your love language?

As it's nearly Valentine's Day, it's a good time to feature this new story on so-called "love languages." According to a new review by relationships researchers, it looks like we should lump this idea, along with the Myers Briggs Type Inventory and learning styles, into the category of "things that make research psychologists cringe."

Many journalists have covered an empirical review article recently published in the journal, Current Directions in Psychological Science. by psychologists. This post features an story for the website medical express.

Even if you don't know your love language, you've probably heard of the concept. The theory's pervasiveness in pop culture has only increased in the 30-odd years since Baptist pastor Gary Chapman published his book "The Five Love Languages: The Secret to Love That Lasts." [...]

"We were very skeptical about the love languages idea, so we decided to review the existing studies on it," says Emily Impett, a professor in the UTM department of psychology who collaborated with UTM graduate student Gideon Park and York University Assistant Professor Amy Muise. "None of the 10 studies supported Chapman's claims."

Chapman's book argues that there are five love languages: physical touch, words of affirmation, acts of service, quality time and gifts, and that each person has a primary preference for one of them. Furthermore, he argues in his book that when two partners have the same love language (for example, both of them prefer physical touch) that their relationship will be better.

The journalist explains that Impett and her team tested each of Chapman's ideas. One was that people only have one primary love language. In fact, Chapman's unscientific "quiz" forces people to choose one. When people are provided a likert scale for each type, they report multiple preferences of showing and expressing love. Second, Chapman's original thinking was based on a restricted sample (mostly White, religious, middle-class Americans), and in other cultural contexts, there may be different ways of expressing love. 

But let's focus on this claim:

Most importantly, Impett and her team found no scientific evidence for Chapman's central contention that people who choose partners that speak their love language, or learn to speak it, will have more successful relationships. "There's no support for this matching effect," says Impett. "People are basically happier in relationships when they receive any of these expressions of love." 

The researchers reviewed the literature and found

no scientific evidence for Chapman's central contention that people who choose partners that speak their love language, or learn to speak it, will have more successful relationships. "There's no support for this matching effect," says Impett. "People are basically happier in relationships when they receive any of these expressions of love." 

This conclusion is a great example of the theory-data cycle. Let's walk through it. I've included Figure 1 for your reference.

a) The theory-data cycle starts with a theory. What is Chapman's theory about love languages? Remember that a theory is a series of general statements about how variables are related to each other. 

b) A researcher can then design a study to test the theory. In one study that Impett and her team reviewed, researchers recruited a sample of couples and measured each person's love language, and also measured their relationship satisfaction. Then they divided the couples into two types: those who had the same love language and those that did not. 

c) The next step in the theory-data cycle is to state a hypothesis for the study--what should the results of this specific study be if the theory is true? Sketch a graph of the predicted outcome, or state specifically what the results should be. 

d) The next step is to evaluate the results in light of the theory. What did the study actually find? Do these results support the theory, or not? 

 

 

Increasing vaccination rates in evidence-based ways

One of the evidence-based techniques was to "say because," as these ads illustrate.
Images used with permission from the Government of Jersey

Jersey is a self-governing island in the English Channel off the northwest coast of France. Its population is around 108,000. After the COVID-19 vaccine became widely available, the island's government started campaigns to encourage people to get their shots.  Rather than making up communication strategies that just seemed right, they decided to design strategies based on social psychological evidence.  This article describes the studies they used and how they applied them. Let's take a look. 

Strategy: Say because

One strategy mentioned in the article involves the word "because."

The study:

Ellen Langer demonstrated the persuasive power of the word “because” in her famous studies in the 1970s. People waiting in line to use a photocopier were asked “Excuse me, I have five pages. May I use the copier?” 60 percent agreed. But when the request was accompanied with a reason — “May I use the Xerox machine because I’m in a rush?”—94 percent complied.

[...] even when the word because was followed with a completely meaningless reason — such as “May I use the Xerox machine, because I have to make copies?” just as many people said yes. The study wonderfully demonstrates the motivational influence of the word “because” and its continually reinforced association with the good rationales that typically follow it.

The application:

But just because people pay more attention to the word “because” doesn’t mean that what comes after is less important. Hence our two-step “because-then-reason” strategy. First, we prominently included the word “because” in our vaccination outreach campaigns. Then we followed the word “because” with a personalized reason meaningful to that particular messenger (and their audience).

The images in this post show examples of how the posters highlighted the word because. 

Questions about this strategy:

a)  Before basing an intervention on an empirical study, we'd better be sure that study was experimental (rather than correlational). Why?

b) What were the variables in Langer's study, and what were the levels?  Do you think that Langer's study was experimental or correlational? What are the clues? 

 

Strategy: Build ownership

The study:

A 2020 mega-study conducted by several dozen researchers led by Wharton School professor Katy Milkman demonstrated how a simple change to a text message increased uptake of flu shots. Reminder messages that included the words “your vaccine is reserved for you” boosted rates by as much as 11 percent compared to standard messages such as “protect yourself by getting a flu shot.”

The application:

Milkman’s study provided the inspiration for us to adopt a similar approach in the emails and texts we sent on behalf of the Government of Jersey to citizens whose turn it was to receive their Covid-19 vaccine. The words “your vaccine” were designed to give readers that sense of personal ownership.

c) What were the variables in Milkman's study? What were the levels of each variable?

d) Do you think Milkman's study was experimental or correlational? Explain your answer

e) Why do you think the journalist described this as a "mega-study"? Follow the link to test your hunch. 

f)  The journalist seems to mention only two levels of the independent variable in Milkman's study (the words “your vaccine is reserved for you” versus “protect yourself by getting a flu shot.”). But in the Milkman et al.'s actual study, there were many more levels. Check the link to the study to find out how many other conditions the researchers tested. 

 

Note to Instructors: Figure 1 of Milkman's study shows the effect sizes and CIs for all of the different interventions they tried. It's a really nice way to introduce these statistical concepts. 

 

Claim: "Drinking any amount of alcohol causes damage to the brain"

Photo credit: Dangubic/Deposit Photos

Here's a link to CNN's coverage of a study on alcohol use and gray matter in the brain. The headline reads, "Drinking any amount of alcohol causes damage to the brain, study finds." Let's consider how well the study supports the causal claim the journalist (or the editor) attached to it.  Here's one of CNN's introductory statements: 

In an observational study, which has not yet been peer-reviewed, researchers from the University of Oxford studied the relationship between the self-reported alcohol intake of some 25,000 people in the UK, and their brain scans.

a) The term, "observational study" in this context is synonymous with "correlational study." What are the key variables in the study? What does the quote above tell you about whether the variables were manipulated or measured? 

Here's the next paragraph in the CNN summary: 

The researchers noted that drinking had an effect on the brain's gray matter -- regions in the brain that make up "important bits where information is processed," according to lead author Anya Topiwala, a senior clinical researcher at Oxford.

"The more people drank, the less the volume of their gray matter," Topiwala said via email.

b) The CNN journalist erroneously wrote, "drinking had an effect on the brain's gray matter".  Let's walk through whether this correlational study here can support the journalist's causal claim. The first criterion is covariance, which is about the study's results. Does the study show covariance?

c) The second criterion is temporal precedence. The journalist doesn't provide information about whether the drinking behavior (variable A) was measured first in time, before brain scans measured people's gray matter (variable B). What are the two potential directions this relationship might go?

d) The third criterion is internal validity. There could be some third variable, C, which is associated with both A and B. See if you can come up with a potential C variable that could reasonably correlate with both amount of drinking and reduction in gray matter. 

You might have noticed that the opening to the CNN summary mentions that the study "has not yet been peer-reviewed". 

e) What happens during the peer review process? Do you think journalists should summarize, for the general public, studies that have not yet been peer-reviewed?

f) The empirical report has not yet been published, but you can read more about the study where it is posted on the preprint site, medarxiv. What are some pros and cons of putting one's research on "arxiv" sites like these? Consider issues such as openness of science, peer review, and public interest. 

 

Thanks again, Stephen Chew, for sharing the example!

Conspiracy reasoning vs. Scientific reasoning (the case of QAnon)

Conspiracy theories, unlike scientific theories, are unfalsifiable. Credit: buecax/Depositphotos

Early in the research methods course you learn about the theory-data cycle (Chapter 1). You also learn about the benefits of scientific reasoning over informal, everyday reasoning such as intuition, experience, or authority (Chapter 2). The QAnon conspiracy theory is a good example of several principles in these two chapters.

The quotes from this post come from this excellent story in The Atlantic, by journalist Adrienne LaFrance. 

In case you have never heard of the QAnon conspiracy theory, it started from a series of posts by an anonymous author named Q, who posted occasionally on anonymous "image boards" such as 4chan. Here is a quick recap by LaFrance:

In its broadest contours, the QAnon belief system looks something like this: Q is an intelligence or military insider with proof that corrupt world leaders are secretly torturing children all over the world; the malefactors are embedded in the deep state; Donald Trump is working tirelessly to thwart them. (“These people need to ALL be ELIMINATED,” Q wrote in one post.) The eventual destruction of the global cabal is imminent, Q prophesies, but can be accomplished only with the support of patriots who search for meaning in Q’s clues. To believe Q requires rejecting mainstream institutions, ignoring government officials, battling apostates, and despising the press. 

Part I. Falsifiability
Let's practice describing one way that conspiracy thinking is different from scientific thinking. One of the defining features of scientific thinking is that scientific claims are falsifiable. That is, scientists make claims/hypotheses that can be falsified  (disproven) by collecting and analyzing evidence. 

Most conspiracy theories a cannot be falsified, in part because they are stated in squishy, ambiguous terms, and in part because they are so multifaceted. As LaFrance writes,

In the face of inconvenient facts, [QAnon theory] has the ambiguity and adaptability to sustain a movement of this kind over time. For QAnon, every contradiction can be explained away; no form of argument can prevail against it.

Here is an example of some QAnon reasoning: 

At one point, Harger [a believer she interviewed] told me I should look into what happened to John F. Kennedy Jr.—who died in 1999, when his airplane crashed into the Atlantic Ocean off Martha’s Vineyard—suggesting that Hillary Clinton had had him assassinated. (Alternatively, a contingent of QAnon believers say that JFK Jr. faked his death and that he’s a behind-the-scenes Trump supporter, and possibly even Q himself. Some anticipate his dramatic public return so that he can serve as Trump’s running mate in 2020.) When I asked Harger whether there’s any evidence to support the assassination claim, he flipped my question around: “Is there any evidence not to?”

a) Why is Harger's theory (above) about JFK Jr.'s assassination unfalsifiable?

b) And, How does the theory's unfalsifiable nature make it more likely that Harger can believe in QAnon? 

c) Think like a scientist now. We can write a falsifiable statement (i.e., one that could be demonstrated by evidence to be wrong) related to JFK Jr. For example, to falsify the theory that "J.F.K. Jr. will become Trump's running mate in 2020" we could predict this statement:

"J.F.K. Jr.'s name will appear on the ballot with Trump in the 2020 Presidential election." (This is falsifiable because if someone else's name appears on the ballot, then this theory is wrong.)

Now see if you can come up with a falsifiable statement derived from the theory (above) that J.F.K. Jr.'s plane crash death was an assassination. Your statement can take the form, If J.F.K. Jr. was assassinated, then we should see __________. 

Part II. Accepting evidence
One feature of QAnon theory that makes it impossible to falsify is that its believers do not accept any evidence from legitimate sources. That is, whereas scientists use independently verifiable data to support or falsify theories, QAnon believers do not accept the legitimacy of such data. Here's LaFrance again:

Q frequently rails against legitimate sources of information as fake. Shock and Harger [the two believers] rely on information they encounter on Facebook rather than news outlets run by journalists. They don’t read the local paper or watch any of the major television networks. “You can’t watch the news,” Shock said. “Your news channel ain’t gonna tell us [expletive].” Harger says he likes One America News Network. Not so long ago, he used to watch CNN, and couldn’t get enough of Wolf Blitzer. “We were glued to that; we always have been,” he said. “Until this man, Trump, really opened our eyes to what’s happening. And Q. Q is telling us beforehand the stuff that’s going to happen.” I asked Harger and Shock for examples of predictions that had come true. They could not provide specifics and instead encouraged me to do the research myself. When I asked them how they explained the events Q had predicted that never happened, such as Clinton’s arrest, they said that deception is part of Q’s plan. Shock added, “I think there were more things that were predicted that did happen.” Her tone was gentle rather than indignant.

d) Why does QAnon's rejection of legitimate sources of information help its followers maintain their beliefs? 

As an aside, in the passage above, pay attention to the believer's statement that "deception is part of Q's plan." This feature virtually ensures that the theory remains unfalsifiable: Either believers will point to evidence that supports the theory, or, when faced with evidence that does not support the theory, they will claim that "deception is part of Q's plan." The theory is supported no matter what. 

Part III.  Confirmation bias.
In Chapter 2, you learn about quirks of human reasoning that make our intuition inferior to the scientific method. One of these quirks is the confirmation bias, the tendency of people to notice and remember evidence that confirms their beliefs, and to ignore or discount evidence that disconfirms them. 

e) How might confirmation bias be at play in the passage above, about events that Q predicted but that never happened? 

Confirmation bias is probably more prevalent when situations and theories are ambiguous. Ambiguity is clearly at play in QAnon, because Q usually makes general (rather than specific) threats. LaFrance writes,

—this is a very welcoming belief system, warm in its tolerance for contradiction—and are also what makes it possible for a practical man like Hayes [who has a youtube channel "explaining" Q's theories] to play the role that he does. QAnon is complex and confusing. People from all over the internet seek guidance from someone who seems levelheaded. 

f) Explain how confirmation bias, rather than scientific thinking (which strives to objectively evaluate evidence both for and against a theory), can explain the following reasoning about who Q himself is:

Many QAnon adherents see significance in Trump tweets containing words that begin with the letter Q. Recent world events have rewarded them amply. “I am a great friend and admirer of the Queen & the United Kingdom,” Trump began one tweet on March 29. The day before, he had tweeted this: “I am giving consideration to a QUARANTINE.” The Q crowd seized on both tweets, arguing that if you ignore most of the letters in the messages, you’ll find a confession from Trump: “I am … Q.”

Coda.
QAnon might seem like a harmless academic example of conspiracy theories. And, as LaFrance suggests, some people follow it just for fun (it might have even started out as a parody). However, as LaFrance argues, it has actually incited violence in the past. For example, in 2017, one believer walked into a Washington, D.C. pizza shop with a rifle because he had believed QAnon message boards that children were being held hostage in the shop's basement (the building doesn't have one). (He was wrong: "'The intel on this wasn’t 100 percent,' Welch told The New York Times after his arrest.")  LaFrance also reports, "In 2018, Reddit banned QAnon groups from its platform for inciting violence."  In the United States, several congressional candidates have promoted QAnon's theories. Unfalsifiable conspiracy thinking might give our leaders a (false) sense of confidence, but it certainly doesn't belong in government discourse.  

If QAnon supporters call for violence, revolt, or collective action and people do not think critically about what they say, this fake conspiracy theory will become a real threat to our democracy. 

Here's a great story of a person who used to believe in QAnon and has subsequently changed his mind. 

 

 

 

 

 

 

 

Racial bias in psychological science journals

A recent study found that in psychology, the authors, participants, topics, and editors of empirical journal articles are mostly White.
Photo credit: Africa Studio/Shutterstock

Is science fair?

You learned in Chapter 2 that empirical journals are the primary place where psychological scientists make their research public. Each empirical journal has an editorial board-- a group of scientists who have power to select the peer reviewers for each article and make the final decision about publication. 

Each step of the way, the stated goals of the publication process are to be fair, to focus on the quality of the science, and to publish the best science possible. But is the process fair?  In light of the renewed energy around anti-racism, a timely new study sheds light on racial diversity in the psychology publication process. The researchers asked, Is race an important variable in the scientific publication process?

The team coded all empirical journal articles that appeared in the most prestigious journals in psychology (over 26,000 articles). There were two journals each from social, cognitive, and developmental psychology. The authors measured:

• how often race was studied as a subject in these articles
• the race of the participants in the studies
• the proportion of the authors of scientific articles were White vs. Non-white
• the proportion of the editors of the journals themselves were White vs. Non-white

Here are some of the results of this published study of published studies, as summarized by the Association for Psychological Science's trade publication:

The results suggest that there is a hierarchy in psychology research that determines who and what gets published. “We are not saying anyone has bad intentions,” said Roberts [one of the study's authors]. “It’s a systemic problem across many fields, and psychological science is no exception.”

First, across the past five decades, articles in psychological journals that highlight race have been rare, and although developmental and social psychology journals have published a growing number of these articles, they have remained virtually nonexistent in cognitive psychology.

Second, most journals have been edited by White editors, under whom there has been a notable dearth of published articles highlighting race and racism.

Third, many of the publications that highlight race have been written by White authors, who employed significantly fewer participants of color.

Let's parse these claims. 

Questions

a) What kind of a claim is it (frequency, association, or cause) to state that "across the past five decades, articles in psychological journals that highlight race have been rare"

[And what's "rare"? The published report says that "only 5% of publications highlighted race" (p. 4). This 5% figure is an average across cognitive journals (less than 1%) and social (5%) and developmental journals (8%)]

b) Let's take the second claim, "most journals have been edited by White editors, under whom there has been a notable dearth of published articles highlighting race and racism"

The quoted text above provides two claims in one sentence. What is the frequency claim in the quoted text? What is the association claim in the quoted text? What are the two variables in the association claim?

c) Now for the third claim, "many of the publications that highlight race have been written by White authors, who employed significantly fewer participants of color."

As with b), the quoted test provides two claims in one sentence, a frequency claim and an association claim. What are these? 

d) In Chapter 1 of the 4th Edition (available Fall 2020), you will read about Merton's norms of science--a set of values that the community of scientists usually endorses and strives  to follow. Here they are, with short descriptions (for a full review, see Chapter 1 or this summary). 

Universalism: Anyone can do science as long as they follow the accepted criteria for science, which are established in advance, are objective, and are accessible to all.
Communality: Share! The process of science, including data, methods, and products, should be free and open for all to use. 
Disinterestedness: The quality of the science should be the only thing that matters, not the prestige of the person or place it was conducted. 
Organized Skepticism: Verify everything. Critical peer review matters, every study should be replicated, and criticism of one's evidence is expected and welcome. 

Roberts and his coauthors argued that psychology is too White. Which of Merton's norms does this result seem to violate, and why? 

Suggested answers:

a) This is a frequency claim--a statement about the frequency of the variable, "highlighting race" across 26,000 journal articles.

b) The frequency claim is about the proportion of journal editors who are White vs. a Person of Color (POC). The association claim is about a link between two variables: whether a journal has a White editor or not, and whether that journal published articles highlighting race or not.

c)  The frequency claim is about the proportion of article authors who are White vs. POC. The association claim is about a link between two variables: whether the article was written by a White author or not, and whether that article included POC as participants or not.

d) Open to discussion!

Here is a summary of this work.

Evidence-based tips for well-being during a pandemic

Dr. Santos provides evidence-based ways to enhance well-being during this pandemic. Credit: Yale University

How can people enhance their wellbeing during the Coronovirus pandemic? 

Dr. Laurie Santos (pictured) has been meeting with several news outlets, sharing evidence-based tips from her psychology of well-being class at Yale University. A version of the course is also offered on Coursera.

In the Twitter-embedded video here, produced by the World Economic Forum, Dr. Santos covers five tips for coping during the pandemic. In this space, I'm going to quote moments from the video, and you can practice classifying the claims and describing how the studies would have been conducted. The first three of these tips are also covered in this story from the World Economic Forum.

Tip 1: Keep social, even when socially distancing

Dr. Santos states in her segment that "Happy people spend more time with other people in general. They also prioritize time with their friends and family members"

a) What kind of claim is it to say that "happy people spend more time with other people in general"--frequency, association, or cause?  What are the variables in this claim? 

b) What kind of study was likely conducted to support this claim--correlational or experimental? 

c) Sketch a scatterplot with well-labeled axes that could depict this result.

Tip 2: Help others

To explain this tip, Dr. Santos mentions, "The research shows that doing random acts of kindness...can be incredibly powerful." For example, happy people are giving money to charity and volunteering in their communities. 

d) What kind of claim is it to say that "doing random acts of kindness...can make you happy"--frequency, association, or cause?  What are the variables in this claim? 

e) What kind of study would need to be conducted to support this claim--correlational or experimental? 

f) Can you imagine how you  might manipulate the variable, "doing random acts of kindness?"

Tip 3: Move--even if you can't leave your house

In this section, she says, "Exercise and moving your body is really correlated with improved well-being"

g) The verb, "correlated" gives this one away: It's an association claim. Dr. Santos specifies that they are "really correlated," which seems to suggest a strong effect size. What factors do we consider when we're trying to decide if a correlation (an effect size) is strong versus weak?

Tip 4: Savor everyday pleasures

"Research suggests that happy people tend to be more mindful. They tend to savor the present moment; they tend to be in the present moment"  Here Dr. Santos also strongly recommends taking deep breaths to calm the vagus nerve and connect to the present. 

h) How might you design an experiment to test the hypothesis that deep breathing can improve people's well-being during quarantine? Whom might you recruit for such a study? How would you operationalize each of the two variables? What results would you expect to find? 

Tip 5: Think about how you want to look back on this time

Finally, she shares one of my favorite tips for emotional regulation, psychological distancing: "Psychological distancing is the idea that if we think about ourselves and how we will have dealt with this crisis in the future...Think about how you want to tell your grandkids you lived through COVID-19, you don't want to be like, 'I panicked-bought and hoarded toilet paper..no, you want the story to be positive...'"

Experimental research has supported the causal claim that self-distancing--seeing yourself as though you are a fly on the wall, or even imagining looking back on yourself from the future--is associated with improved emotional regulation. It helps us take a cooler, less impulsive view of our emotions and process them instead of ruminating. For example, self-distancing helps people manage feelings of anger in constructive ways (here's a nice summary). 

i) Review the term translational research in your textbook (Ch 1) and explain how we might conduct translational research on psychological distancing during the pandemic. What is the basic research it is based on? What is the real-world context it's being translated to? And what kind of study could translate--that is, bridge the gap--between the two?