Everyday Research Methods

Amazon's Alexa and baby-naming trends

Amazon's choice to use the wake word "Alexa" seems to have led to a decrease in that name's popularity for babies. Photo: vectorstockcompany/Depositphotos

Here's a report from the Washington Post, entitled "Amazon, can we have our name back?". The journalist interviewed several women named Alexa, reporting about their experiences related to having the same name as the popular digital assistant. For example, 

Alexa Morales wore her name proudly. But after Amazon launched its voice service, also called Alexa, in November 2014, people began speaking to Morales differently. She said they made jokes about her name, giving her commands or asking her questions in a robotic tone. “When I hear my name now, it’s not good thoughts, it’s like, tensing,” said Morales, 28, a pharmacy technician and student in Bridgeport, Conn.

Much of the article is dedicated to the negative experiences (including harassment) that people with the name Alexa have had. Several Alexas have chosen to adopt nicknames. One psychologist was quoted as saying,

“They’re sort of growing up with the communication that their name is associated with a servant role,” Christopher Kearney, a professor of psychology at the University of Nevada at Las Vegas, said in an interview. “If a teacher is saying something derogatory, even if it’s just as a joke, or it’s to change their name, or … allows other kids to participate in that, it can create a lot of psychological damage.”

Aside from these detailed experiences, one part of this article can be used as an example of an interrupted time-series design. 

Nearly 130,000 people in the United States have the name Alexa. It gained popularity after singer Billy Joel and model Christie Brinkley named their daughter Alexa in 1985. In 2015, more than 6,000 baby girls in the United States were named Alexa, according to a Washington Post analysis of Social Security Administration data.

After Amazon chose Alexa as the wake word of its voice service, the name’s popularity plummeted. In 2020, only about 1,300 babies were given the name. 

It's much easier to think about this result if you see the graph of it, so I urge you to visit the Washington Post story and scroll down to the line graph before reading on. 

a) In this interrupted time series design, what is the quasi-independent variable? What is the dependent variable?

b) What do you think--to what extent can the baby name data in the article support the causal claim that "Amazon's Echo devices caused a decrease in people naming their babies Alexa"?

Specifically, the graph shows covariance and temporal precedence--Amazon's Alexa device is associated with a subsequent, dramatic drop in the number of babies named Alexa. But what about internal validity--are there alternative explanations we can consider for this drop? 

c) What are your thoughts after reading the article? Do you that Amazon should change the Echo's default "wake word?"

 

 

 

 

Music festivals "make people more generous, socially connected"

Credit: halfpoint/Deposit Photos

Summer is the time for music festivals like Burning Man, Burning Nest, Firefly, and Bonnaroo. If you are planning to attend one this year, this post might interest you.

The science journalism clearinghouse called Study Finds recently reported on a study conducted at Yale University. The journalists' report included the causal headline that Music festivals like Burning Man make people more generous, socially connected to others". 

a) What are the variables in this causal headline, and what makes it a causal claim?

b) In general, researchers need to conduct an experiment in order to support a causal claim. Do you see any barriers to conducting an experiment on the causal effects of music festivals?

It will be interesting to read about whether the researchers' study can feasibly support the causal claim that the journalist attached to it. Here's how Study Finds described the study's method: 

The Yale researchers studied the behavior and experience of over 1,200 people attending festivals such as Burning Man, Burning Nest, Lightning in a Bottle, Dirty Bird, and Latitude. All of the festivals took place in the United States or the United Kingdom and emphasized art, music, and self-expression. The team set up booths at all the events, which they called “Play Games for Science.”

People ... self-reported their experiences at the event and whether they were willing to share their resources with friends and strangers. For example, week-long festivals such as Burning Man do not sell food, and people who find themselves short must rely on the generosity of others.

[...] Overall, 63.2 percent of people said they had a transformative experience and felt themselves changing. ... One of the takeaways from the transformative experience was feeling more connected to other humans, which made them more likely to share food and resources with other attendees.

c) So far, do the method and results of this study (as reported by the journalist) support the conclusion that attending the festivals caused a transformative experience? Why or why not? (Hint: Think about the need for a comparison group.)

d) Based on what you've read, was the variable of "transformative experience" operationalized as a physiological, observational, or self-report measure?

In this case, the journalist's coverage of the study doesn't do the original study justice, as I discovered when I read the original empirical article, which appeared in the journal Nature Communication (you can access it here).There are several important omissions in the journalist's report. One is that the researchers themselves did not use causal language in their work. Instead, their article is titled "Prosocial correlates of transformative experiences at secular multi-day mass gatherings" (my emphasis).

The Study Finds journalist also omitted key methods and variables. For example, one important variable in the original study was that the researchers graphed people's self-reports of having a transformative experience on each day of a 7-day festival (see Figure 2,  here). Using this method, the researchers found that the later the day (going from Day 1 to Day 7), the more likely respondents were to report having a transformative experience. 

e) Why is the above method and result important, in terms of supporting the claim that the music festivals can increase people's tendency to have a transformative experience? (Hint: think about temporal precedence.)

Another important detail omitted by the Study Finds story, is that in addition to asking people to self-report their transformative experiences, researchers also asked participants to imagine playing a "dictator game". In this game, participants are given 10 valuable tickets. Participants can then donate some number of their tickets to randomly selected stranger. The dictator game is an observational measure of generosity to strangers. The researchers found that festival participants chose to donate between 50 and 75% of their tickets to the stranger. This was much higher than the level of donations typically found, as reported in a meta-analysis of all previous studies using the dictator game (which found an average of 26%). 

f)  This dictator game result is a type of quasi-experiment--we might call it a non-equivalent control group, posttest-only design. What is the quasi-independent variable--that is, what are the two levels of the non-equivalent control group (and why are they "non-equivalent")? What is the dependent variable? 

g) The result suggests that attending the music festival might cause people to be more generous in the dictator game, but there might be a selection effect. Why? 

 

Where do you get more creative ideas?

The research tested whether people came up with more, and more creative, ideas when working face-to-face compared to Zoom.
Photo credit: MBI/Alamy Stock Photo

Apparently, researchers have found that Zoom can stifle pairs' creativity. Here's how an empirical study on this topic was introduced by journalists writing for The Guardian:

While the benefits of Zoom and other videoconferencing tools made them indispensable in the pandemic, the research suggests that heavy reliance on the technology comes at a cost to creative thinking.

a) Before moving on, how might you design an experiment to test which type of meeting would be better for creativity? What would your study's independent variable (IV) be? What would the dependent variable (DV) be? 

Let's see how the researchers did it. According to the journalist,  

... the researchers recruited more than 600 volunteers who were paired up to tackle a creativity task either together in the same room, or virtually over Zoom.

The pairs had five minutes to come up with creative uses for a Frisbee or bubble wrap and a minute to select their best idea. Independent judges ruled that turning a Frisbee into a plate was less creative than using it to knock fruit from a tree, while using bubble wrap to send morse code messages was more creative than using it to protect a baby. Overall, those who worked over Zoom had 20% fewer ideas than those who met face to face.

b) In the study above, what was the IV and what were its levels?

c) Was the IV manipulated as between groups or within-groups?

d) What was the DV and what were its possible levels?

e) What simple experiment is this: Posttest-only? Pretest-posttest? Repeated measures? Concurrent measures? 

Now here's a second experiment: 

The same effect was apparent in the real world. In a field study, the researchers analysed ideas for new products generated by 1,490 engineers for a multinational company. The engineers, who were in Finland, Hungary, India, Israel and Portugal, were randomly paired up and given an hour or so to brainstorm products either in person or over Webex videoconferencing. They then selected their best idea.

...the researchers report that the engineers produced more ideas, and more innovative ideas, when working face to face. “They are not only generating a larger number of creative ideas, but their best idea is better,” 

f) This experiment was billed as a "field study." What does that mean? 

g) In the study above, what was the IV and what were its levels?

h) Was the IV manipulated as between groups or within-groups?

i) What were the two DVs in this study?

j) What simple experiment is this: Posttest-only? Pretest-posttest? Repeated measures? Concurrent measures? 

k) The second study is a replication of the first one. What role does replication play in the scientific process? If you've studied Chapter 14, you could identify whether this was a direct replication, a replication-plus-extension, or a conceptual replication.

Interestingly, the journalist notes, "it is unclear whether the impact on creativity holds for larger teams." This statement is a good example of external validity as it applies to situations (rather than the most common focus of external validity, which is on a study's generalizability to participants).

l) What other situations might we wonder about as we consider the external validity of this research? 

The original empirical study was published in Nature. The original study contains a number of interesting additional DVs, including potential mediators for the overall effect on creativity. 

How much weekly exercise can prevent depression?

The weekly recommendation for exercise is 150 minutes.
Photo credit: blasbike/Deposit Photos

A journalist wrote about a recent meta-analysis in JAMA Psychiatry which evaluated the connection between weekly time spent exercising and levels of depression. The headline of the Washington Post summary read, "Exercising even half the recommended amount can help prevent depression". Let's delve into this research, as covered by the journalist in their brief report. You can access the original empirical work here. 

Here's the opening message:

Already known to help ease depression, regular exercise may also help prevent it, with people who exercised just half the recommended weekly amount lowering their risk for depression by 18 percent.... However, those who were more active, meeting at least the minimum recommended physical activity level, reduced their risk for depression by 25 percent, compared with inactive people.

The empirical research was a meta analysis, and it seems that all of the studies involved longitudinal designs: 

The findings stem from the analysis of data from 15 studies, involving 191,130 adults who were tracked for at least three years.

a) What are the steps you'd typically follow if you were conducting a meta-analysis? 

According to the journalist, the recommended amount of weekly exercise is 150 minutes. Therefore, it seems that working out only 75 minute per week (by walking, for example) does predict lower rates of depression over time. 

Let's apply the four big validities to this study. 

b) Construct validity: We'd need to address the construct validity of each major variable--exercise and depression. It's likely you can measure exercise in multiple ways (self-report, observational, physiological). Can you tell from the journalist's report how the studies measured exercise? 

What about depression--can you tell from the report how the studies measured depression?

I would assume that across the 15 studies, there were a variety of ways that depression and exercise were measured. Is this variability of operationalizations a strength of the meta-analysis, or a weakness? 

c) Statistical validity. The journalist merely reports a single estimate:  "people who exercised just half the recommended weekly amount lowering their risk for depression by 18 percent." But the empirical article gives us a 95% CI for this estimate: [13%, 23%].  In addition the journalist writes, "meeting at least the minimum recommended physical activity level, reduced their risk for depression by 25 percent, compared with inactive people." Again the empirical article gives the 95% CI as [18%, 32%%]. 

In your own words, explain what these two 95% CIs mean.

d) External validity. As we consider external validity, you might be tempted to focus on the total number of participants in the meta-analysis (more than 191,000 over 15 studies). Why would it be incorrect to use sample size to make decisions about external validity? 

e) Internal validity. All 15 studies in this analysis were prospective, or longitudinal. What does that mean? Use the term "temporal precedence" in your response.

Now to focus on internal validity specifically: Can you think a possible third variable that might be associated with both exercise and depression over time? (It would be potentially important for the researchers to control for this variable if they hoped to support a causal claim.)

 

 

 

 

 

 

Correlates of depression in boys and girls

The study found a moderated relationship. What are the three variables involved? Photo: cozyta/Deposit Photos

This story comes from the website studyfinds.org--a website that covers a wide range of scientific studies (and therefore a great place to practice critical reading of how journalists cover science news).

The StudyFinds journalist describes a newly-published Dutch study on body image and depression in teenage boys and girls. The result is an example of a moderator, so let's practice moderator reasoning.

Here is some of the journalist's description:

The study by researchers at Erasmus University in The Netherlands tracked 4,000 teenagers for several years. Researchers found that young men are a staggering 285 percent more likely to have a severe depressive episode if they were unhappy with their bodies as teens. 

Their female counterparts were also affected, but were 84 percent more likely to suffer the same problems if they had been insecure about their appearance. 

You might be wondering what it means to be "84 percent more likely" or "285 percent more likely?"  These numbers seem huge. You should convert the percentage to a decimal (i.e., 0.84 or 2.85), and then multiply that times some baseline rate. So in the case of depression, let's imagine that for girls, the baseline rate of depression diagnosis is 10 (out of 100)--we'd then assume that the risk for girls who are insecure about their appearance would have a rate of about 18.4 (out of 100). For boys, the baseline rate of depression might also be 10 (out of 100); we'd calculate that the risk for boys who are insecure about their appearance would have a rate of depression diagnosis of about 28.5 (out of 100). 

a) It takes three variables to have a moderator--two for a core bivariate relationship and one more variable as the moderator. Identify the three players in this example:

Core bivariate relationship (2 variables):
Moderator variable:

b) In this example, are the variables manipulated or measured? 

c) We can depict a moderator two ways: with a table or with a figure. Let's do a table first--this will be  similar to Table 8.5 and 8.6. Note that you don't have to use the numbers provided (285% or 84%)--just think about how strong the relationships are, relative to each other. 

On each line, enter one level of the moderator variable	What is the core bivariate relationship and how strong is it? (strong, weak, or zero)

d) We can also depict a moderator with a graph. Sketch a pair of bar or line graphs. Label one graph for boys and one for girls. On the x-axis, put the "body dissatisfaction" variable; put the other variable on the y-axis. If you are making line graphs, use steepness to depict the strengths of the two relationships (which graph will be steeper--boys or girls?). If you are making bar graphs, use the difference between the bars to depict the strength of the relationship (which will be stronger--boys or girls?). 

e) Now write a sentence similar to those in Figure 9.14: "_________ moderates the relationship between _______ and _________, such that ___________."

 

Refining your understanding: A moderator is a relationship changer: it changes the strength or direction of some core bivariate relationship. In the present example, it's also true that there were gender differences in one of the variables--relative level of body dissatisfaction: 

Well over half of teens, 61 percent, experience body dissatisfaction.... Nearly one in three girls and around one in seven boys were insecure about their weight, and around one quarter girls and one in seven boys were dissatisfied with their figure. 

Sometimes students confuse simple differences (i.e., boys are lower than girls on body dissatisfaction) with moderators (i.e., boys with body dissatisfaction are more vulnerable to depression than girls are). Take a moment and study the difference. How many variables are involved in each statement:

boys are lower than girls on body dissatisfaction
boys with body dissatisfaction are more vulnerable to depression than girls with body dissatisfaction are

f) Challenge yourself to create a scatterplot in which body dissatisfaction is on one axis and risk of severe depression is on the other. Use one color of dot to represent boys; pick another for girls. Carefully place dots on this scatterplot to represent the two major findings--that boys are generally lower than girls on body dissatisfaction, AND that boys who do have body dissatisfaction are more at risk for severe depression. (Consult Figure 8.17 in the 4th edition for a model.)

Therapy dogs in the emergency room

The study focused on therapy dogs' effect on perceptions of pain in emergency room patients. Photo: monkeybusiness/Deposit Photos

Adults and children alike can benefit from petting a therapy dog (a friendly pet who makes rounds with a trainer, perhaps to a clinic, pediatric unit, or even on campus during finals week). In this study, covered by journalists at NPR news, researchers from the University of Saskatchewan tested the efficacy of therapy dogs for people who visited an emergency room (ER) complaining of pain. 

 

Here is how the journalist described the study: 

In a controlled trial, researchers asked more than 200 patients to report their pain levels. One group of patients received a 10-minute visit from a therapy dog and the other group did not. After the dog visit, researchers asked patients in both groups to report their pain levels again.

Those who spent 10 minutes with the dog reported less pain [over time], the study found.

Based on this simple description, you can assume that this study was an experiment. 

 

a) What was the independent variable (IV)? What were its levels?

b) Was the IV manipulated as independent groups or within groups? 

c) What was the dependent variable (DV)? 

d) Which of the designs in Chapter 10 does this represent: Posttest only? Pretest-posttest? Repeated measures? or Concurrent measures? 

e) Sketch a small graph of the study's outcome. You don't know what the exact values are, but you should be able to estimate what should be higher and lower. Use either a bar graph or a line graph, following examples in Chapter 10. 

 

The journalist states, "A new study shows that a 10-minute visit from a therapy dog can help relieve patients' pain in the emergency room."  

f) What makes this a causal claim?

b) Assuming that people were randomly assigned to the two groups. Can the study support this causal claim? Name and apply the three causal criteria. 

 

You can read the real study for free: The empirical article is open access in PLOS. 

Psychological consequences of a "nearly natureless world"

Humans who can see nature views from their windows tend to be happier. 
Photo: © Amla Sanghvi

The examples in this post come from an extended piece by journalist Alan Lightman. He writes about how, 

For more than 99 percent of our history as humans, we lived close to nature. We lived in the open. The first house with a roof appeared only 5,000 years ago...

Lightman argues that in our modern lives, most of us spend most of our time working on computers and interfacing through screens, and this shift has potential consequences for our well-being. 

One of the psychological concepts the journalist uses as evidence in this article is known as "connectedness to nature." Two social psychologists developed a connectedness to nature scale (CNS), which is a self-report questionnaire with multiple items on it, including

"I often feel a sense of oneness with the natural world around me"
"I feel as though I belong to the Earth as equally as it belongs to me"
"I think of the natural world as a community to which I belong"

People who score high on this scale are said to have more connectedness to nature. 

Here's how the journalist describes one line of research with the CNS:

In recent years, psychologists have undertaken a number of studies to investigate correlations between scores on the CNS test and well developed methods for measuring happiness and well-being. In 2014, the psychologist Colin Capaldi and his colleagues at the Public Health Agency in Canada combined 30 such studies, involving more than 8,500 participants. The psychologists found a significant association between nature connectedness and life satisfaction and happiness. 

a) The quote above describes a meta-analysis. Can you see why? 

b) If you were to sketch the association found in Capalidi's meta-analysis, how would you do so? Sketch a scatterplot, labeling the axes mindfully.

c) Is the research above considered correlational or experimental? 

Here are two quasi-experimental studies that the journalist reviewed: 

Hospital patients in rooms with foliage or windows looking out on gardens and trees do better after surgery. Workers in offices with windows that open up to pastoral-like views have less anxiety, more positive work attitudes, and more job satisfaction.

d) Pick one of the studies above and identify the quasi-independent variable and the dependent variable(s) in the study. Sketch a small bar graph of the results described.

e) What makes these studies quasi-experiments instead of true experiments? 

e) Given the studies described above, come up with your own nature-related hypothesis to test. For example, what else, besides well-being, might the CNS be correlated with? Or, what other variables, besides hospital views and office views, might be worth studying with a quasi-experiment?

 

 

Does psychopathy predict success?

The study empirically tested the widely-held hypothesis that psychopathy would be associated with more career success. Photo: Mangostar/Shutterstock

It's a common belief that to get ahead in business or politics, people have to be driven, callous, and unsympathetic. Is it true? A journalist for Forbes magazine reviewed a study by personality psychologist Hedwig Eisenbarth. The study used methods you're learning about in Chapters 8 and 9.

The journalist introduces the topic with an interview with the researcher: 

“There is this public opinion or belief that psychopathy is related with higher success — that individuals high on psychopathic traits would be successful CEOs or politicians,” says Hedwig Eisenbarth, ...lead author of the research. “People often think so because they associate the cold-bloodedness, fearlessness, and low empathy that psychopathy is characterized by with high income positions....we wanted to find out if this actually has an empirical basis.”

The journalist reported that the study used a representative sample of New Zealand adults. The sample size was 2300. 

The researchers measured three key variables: psychopathy, professional success, and occupational prestige. Psychopathy was operationalized via self-report responses to surveys that measured "fearless dominance, self-centered impulsivity, and coldheartedness." Professional success was operationalized via self-report measures of job satisfaction and job security. And occupational prestige was operationalized as a combination of socioeconomic status and the status of their job. 

The results went against the prevailing view that psychopathy is associated with more job success. Here's the journalist's summary:

From these measures, the researchers were able to test their core hypothesis — that people with psychopathic personality tendencies would be less likely to achieve high levels of professional success. They found this to be mostly the case. Individuals higher on the fearless-dominance aspect reported higher subjective professional success while individuals higher on self-centered impulsivity reported lower subjective professional success. Furthermore, those higher on coldheartedness showed lower occupational prestige. 

Questions

a) The paragraph above reports three bivariate correlations. You can sketch each one of them on its own scatterplot. Do that now--use the text to decide what variables go on each axis as well as what the slope of the scatterplot should be.

b) Note that the journalist describes above the direction of the bivariate correlations, but does not mention the strength of the relationships. How spread out were the dots on your sketched scatterplots for a)? The spread of the dots is an indicator of how strong you think the relationships might be. Make adjustments if necessary.

The journalist reported a regression result, as well:

These associations held true even when the researchers accounted for other variables that might have influenced their predictions, such as gender, education level, age, and duration of job tenure.

c) Why might it be important to control for the variable of gender in these relationships? Explain how one gender might be higher on both of the variables in each scatterplot. 

 

Swapping saliva

According to the study, even babies know that we don't share our food with just anybody... Photo: szefei/Shutterstock

When you see the photo here, you probably assume the kids are siblings, or at least very close friends. It's not the matching dresses--it's because they're taking turns tasting the same ice cream cone--an activity in which they'll undoubtedly exchange saliva. We humans tend to conclude that people willing to share spit with each other are probably more intimately related. 

Do babies realize the same thing? Apparently so! Here's an NPR story about a research study on whether babies also conclude that people who share saliva  also conclude that they are more closely related. They introduce the story like this: 

"From a really young age, without much experience at all with these things, infants are able to understand not only who is connected but how they are connected," says Ashley Thomas of MIT, who studies what babies and young children understand about the complexities of their social world. "They are able to distinguish between different kinds of cooperative relationships."

As is usually the case, researchers have to get creative to study what young kids think. The present set of studies tested both infants (aged 8-10 months) and young toddlers. They used a puppet show to experimentally manipulate different variables: 

One of their videos shows a woman rolling a ball back and forth with a blue fuzzy puppet. Then another woman shares an orange with that same puppet by putting a slice of orange in her mouth, then letting the puppet nibble on the slice, and then putting it back in her own mouth.

Then they used looking time to measure what infants were thinking:

...the video then shows the two women with the blue fuzzy puppet in between them. The puppet starts to cry and puts its head down, as if it is suddenly unhappy.

When the puppet cried, infants and toddlers looked first and looked longer at the woman who had shared bites of her orange.

"They're looking in that direction because they expect something to happen there," says Thomas. "They expect that woman to be the one to respond to the puppet's distress."

Questions:

a) What is the independent variable (IV) in this experiment? What were its levels? Is it manipulated as independent-groups or within groups? 

b) What is the dependent variable (DV)?

c) Which of the four simple experiments does this seem to be--pretest-posttest? posttest only? concurrent measures? or repeated measures?

d) Sketch a bar graph of the results, putting the IV on the x-axis and the DV on the y-axis. 

Here's another part of the study: 

When the two women were shown with a totally new puppet that started to cry, however, infants and young children looked at both women equally often. This suggested that they didn't see this particular food-sharing woman as especially helpful; instead, her relationship with the puppet was what really mattered.

e) Sketch a bar graph of these results, putting the IV on the x-axis and the DV on the y-axis. 

Like most modern research articles, this one includes multiple studies (three studies, in fact). Additional studies help provide replications of the observed effects, and can also help rule out alternative explanations.  Here is the journalist's description of one of the other studies in the empirical journal article:

To make sure it wasn't just sharing of food that seemed to make babies infer the existence of a close social connection, the researchers created another, similar video. This time, instead of sharing an orange slice, a woman simply put her finger in her own mouth and then put it in a purple puppet's mouth, before putting it back in her own mouth.

Then that same woman also interacted with a green puppet, touching its forehead and then touching her own forehead. After that, the video showed the woman seemingly in distress, with the purple and green puppets looking on.

Infants and toddlers gazed at the purple puppet that had the more intimate, finger-in-the-mouth interaction, as if expecting this puppet to be more affected by the woman's consternation, presumably because they seemed to have a closer relationship.

f) What are the IVs and DVs in this new study?

g) Sketch a bar graph of the results being described.

h) What type of replication is this: a direct replication, a conceptual replication, or a replication-plus-extension? 

i) One possible explanation for the infants' looking behavior in this study is that infants just prefer to look at purple puppets more than green puppets. How could the researchers rule out this alternative explanation? 

Want to read more? here's a link to the empirical journal article, which appeared in the journal Science.

Curbing hate speech with empathy

The study found that empathy-inducing responses like this were more likely to be followed by people deleting their original hateful posts. Photo courtesy of the author

Hate speech abounds on certain corners of social media. Some people post racist, sexist, anti-immigrant, or homophobic comments, and these contribute to a harmful and hateful discourse online. Some companies filter, censor, or censure accounts when they cross the line. Other companies have proved unwilling to ban such content, citing free speech concerns. Recently, some researchers asked if people might engage with such posters online in order to prevent them from posting hate speech. 

The website Optimist Daily ran a story about their work. Here's how they introduced it: 

Led by public policy professor Dominik Hangartner, the research team identified 1,350 English-speaking Twitter users who had posted racist or xenophobic content. The researchers then assigned these users randomly to a control group or one of three test groups. The test groups were then prescribed different counterspeech strategies: empathy-eliciting messages, humor, or warning of consequences.

For example, the empathy eliciting messages might have read "your post is very painful for Jewish people to read...", or "For African-Americans, it really hurts to see people use language like this". Another example can be seen in the comment about immigrants in the photo above, accompanying this post.  

The humorous messages included a meme of an animal with a quip such as "please sir, stop tweeting".

The warning-of-consequences messages read, for example, "Hey, remember that your friends and family can see this tweet too." 

What happened after one of these responses was posted? Here is how the journalist described the results: 

The researchers found that by far, the most effective strategy for getting these users to delete or amend hateful content was counterspeech messages that elicit empathy.

Questions

a) What words in the journalist's story convey that this was an experiment?  I would additionally classify it as a field experiment--can you see why?

b) What was the independent variable? What were its levels? Was the IV manipulated as independent-groups or within groups?

c) There are two dependent variables mentioned in the quotes above. What are they? If you were the researcher, what other DVs would you be interested in? (you can consult the original study to see some of the other DVs)

d) How were the participants in this study selected? Did they use a random sampling method? Or something else? 

e) This study was approved by the researchers' IRB, but the participants in this study apparently did not provide informed consent to participate in the research. Why do you think the IRB approved the study?

Here is a link to the original journal article in PNAS. You can use it to study the graphs of the study's outcome.