Everyday Research Methods

A serendipitous study of long-term memory in birds

Here's a photo of a toutouwai (North Island Robin) sporting its bands for identification. Photo: Shutterstock

We don't include many animal behavior studies in ERM blog, but here's a great one about animal memory. How long do you think wild animals can remember new skills they have learned? In this report in The Scientist, we read that the North Island Robin (known in the Maori language as toutouwai) of New Zealand can apparently remember a skill up to 22 months, without any reminders in the meantime. Here's how the study worked. 

[Originally, scientist Rachel Shaw] demonstrated that wild toutouwai living in the Zealandia Wildlife Sanctuary in Wellington, New Zealand, are capable of learning a novel task. Using a rectangular box with three compartments, each covered by a lid that swivels to the side to reveal its contents, she incrementally taught the birds how to peck the lids aside and retrieve mealworms hidden inside two of the three wells.

(By the way, the story in The Scientist includes a photo of the experimental apparatus and a video of a bird using it.)

Roughly a year later, Shaw was giving a public talk at the sanctuary about conservation and her work with birds. As she explained the memory apparatus she had used in her previous research, a toutouwai appeared.

“I haven’t given this bird this task in over a year,” she recalls telling the audience, “so let’s see how he does.” As if on cue, the bird instantly flew down to the apparatus and started opening the lids. That got her wondering: how many of the other birds could repeat the memory task and how long could they remember what they were taught?

This serendipitous event prompted Dr. Shaw to design a study to find out. Before reading on, how would you study this question? 

In the new study, Shaw and her coauthor, Annette Harvey, tested 32 of the initial 64 birds that were trained in 2015 and 2016 and banded for individual identification, 30 of which performed the memory task by spontaneously pecking the lids and opening them on their first try. None of the trained birds had seen the box between the initial training and retesting, though some of the untrained control birds had encountered the apparatus in a previous research project. The time between when the birds had first learned the lid-opening behavior and subsequent testing ranged from 10 to 22 months. By contrast, the 17 untrained birds were unable to complete the task.

The article mentions several other examples of long-term memory in animals, including pigeons, tortoises, nutcrackers, and chickadees. Read more if you are interested!

Questions

a)  This study might be considered a simple observational study but it might also be classified as a quasi-experiment. What do you think?

b) What role does the untrained comparison group play in this research? 

c)  Do you think the effect size in this study would be considered small or large? Explain your answer.

d) Chapter 6 describes important construct validity considerations for observational studies, such as reactivity, observer bias, and observer effects. Which of these might be relevant in this study on toutouwai? 

Thanks to my friend Dr. Stephen Chew for sharing this story on Twitter!

Racial stereotypes behind how we explain school shootings

Participants and newspapers were more likely to blame video games when a school shooter was white than when he was black. Photo: Richard Levine / Alamy Stock Photo

What causes mass shootings? Of course, we have no experimental evidence on this question. Correlational evidence (both within the U.S. and evidence that compares the U.S. to other countries) is consistent with the argument that gun availability is a factor. 

However, this post isn't about what really causes school shootings. It's about what people think causes them. And it turns out that the race of a school shooter influences what people believe about the shooter. 

After some school shootings (a subset of mass shootings), people have sometimes looked for answers not in the availability of guns, but in video games. Now, there is actually no evidence linking video games to mass shootings. However, this ABC news story talks about when people are more likely to think that video games are responsible: 

There's little evidence showing a relationship between violent video games and school shootings, but video games are often blamed for overall violence in our communities. A recent study published in the Psychology of Popular Media Culture set out to find if we look for an external explanation for violence by whites more often than we do for African Americans.

"We try harder to make excuses for white perpetrators," said investigator Dr. James Ivory, a professor in the Department of Communications at Virginia Tech, in an interview with ABC News.

The first study gave people a mock news story about a school shooting. When asked what caused the shooting, people were more likely to blame video games if the shooter was white than if the shooter was black.

It turns out that in the original study (linked here;  if it's paywalled you may be able to access via your university library), participants read the mock news story and were randomly assigned to one of two photos of the perpetrator--a mugshot showing either a Black or a White man. 

a)  There were two variables in this study. Name each one. For each, was it manipulated or measured?

b) What kind of study is this, correlational or experimental?

c) Sketch a graph of the results of this study. What will you put on the y-axis? 

d) In the study above, what role does random assignment play? Why did the researchers randomly assign people to see either a Black or White perpetrator?  

 

Here is how ABC news described the second study:

The second study looked at thousands of news articles -- video games were mentioned more often in stories of white shooters [compared to black shooters]. 

e) What are the variables in the study above? Name each one; for each, was it manipulated or measured?

f) What kind of study is this, correlational or experimental?

g) Sketch a graph of the results of this study.

h) In the study above, the researchers could not randomly assign news article authors to write about a black vs. a white perpetrator. How does that change the impact of this study? 

i) Which of the two studies do you think provides the strongest evidence that race influences people's willingness to think video games are a cause of the shooting?

 

Suggested answers

a) One variable is race of the perpetrator (levels are Black or White), this was manipulated. The other variable is "likelihood to blame video games". This was measured.

b) This was an experiment because one variable was manipulated. 

d) Random assignment contributes to the study's internal validity. They can be sure that the same kinds of participants were in each group (reading about Black vs. White perpetrators). Therefore, people who play a lot of video games themselves were equally likely to be in the two groups. You might be interested to know that in the original study, the researchers also pretested the Black and White photos they used to be sure that the faces differed on race but not on perceived attractiveness or dangerousness. This pretesting also contributed to the experiment's internal validity.

e) The first variable was race of the perpetrator in the article (Black or White), which was measured. The second variable was whether or not the article mentioned video games (yes or no).

f) This was a correlational study because all variables were measured. 

h) Because this study used naturally-occurring crimes, the researchers may have an internal validity problem. It is possible that the Black and White perpetrated shootings differed not just on race but also on number of victims, age of victims, geography, or some other variable. This introduces an alternative explanation for the result--perhaps people blamed video games for White perpetrated shootings because they had more victims, had older victims, and so on.

i) You might have selected the first study as the strongest because it was an experiment and allows us to make a causal inference. Or you might have selected the second study because it was conducted on "real world" stimuli such as news stories--therefore it appears to have good ecological validity or external validity. 

Thanks to Marianne Lloyd for this idea!

Did the show "Thirteen Reasons Why" cause an increase in suicides?

The study used a quasi-experimental design.  Credit: Beth Dubber/Netflix/Kobal/Shutterstock

Update 7/19/2019: Netflix has decided to remove the scene depicting the main character's actual suicide, perhaps in response to data from this and other studies. Here's one story on the change. 

The Netflix series "Thirteen reasons why" tells the story of a teenage girl who commits suicide, leaving behind audio recordings about people who contributed to her decision. When the series was released in April, 2017, mental health and school professionals became concerned about the possibility that young viewers of the show would experience increased risk of suicide themselves. The phenomenon of "suicide contagion" occurs when a publicized suicide leads to other suicide attempts. Young people may be particularly susceptible to its effects. Suicide is the third leading cause of death among American teenagers (ages 12 to 19, according to the CDC). 

This Mashable journalist covers an empirical journal article published in JAMA Psychiatry in 2019.  The Mashable story reports: 

A new study, conducted by different authors, came to a similar conclusion: In the three months after 13 Reasons Why launched, the youth suicide rate for 10- to 19-year-olds rose unexpectedly by 13 percent. Based on historical trends dating back to 1999, 94 more American adolescents and teens died by suicide during the three-month period than otherwise expected. Similarly, the study published last month looked at suicide deaths in the nine months following the show's launch and found an additional 195 fatalities. 

Students and instructors will find it useful to consult the publicly-available JAMA Psychiatry article that contains graphs of this study. Specifically, the solid orange lines in Figure 2 show the suicide rates for 24 months of time: 15 months before the show debuted, and about 9 months after. 

a) This study used a quasi-experimental design. Which of the designs from Chapter 13 was it? (Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design?)

b) In the quote above, the journalist reported that an 13 percent increase in suicide equals an additional 94 adolescent teen deaths. This figure captures the effect size of the study's result. What do you think of this effect size? Would you call it small or large? 

The empirical journal article also documents the publicity the show received via social media. Here's how the journalist describes the evidence: 

Niederkrotenthaler and his co-authors looked at potential exposure to the show between April 1 to June 30, 2017, when the first season of 13 Reasons Why had just debuted and was most popular on Twitter and Instagram. Using social media data as a proxy for understanding peak interest in the show, the researchers suggest the show's portrayal of suicide created a 'contagion effect' The phenomenon can happen when media coverage of suicide leads to suicide clusters. 

Students and instructors can consult the empirical article's Figure 1 to see the results of the social media analysis. 

b) Why did the authors of the JAMA article include the social media piece? How does this evidence help them make their argument? 

The researchers cannot establish for certain that viewing the show caused the increase in suicide. Here's the Mashable journalist: 

While the researchers couldn't determine whether the children who took their own lives watched the series, an accompanying editorial written by researchers at Harvard and Stanford argues the study provides "strong evidence" that the Netflix show may have led to that increase. 

"One cannot draw definitive causal conclusions from such data, but that no similar increase was seen in suicide rates other than for the age group to which the media portrayal pertained provides some further compelling evidence that the excess suicides may indeed have been owing to the series," the editorial's authors wrote. 

c) The quote above adds an additional detail about the study. The researchers not only tracked the suicide rates of people aged 10-19; they also tracked suicide rates of people in the 20-29 and 30+ age group. The researchers found an increase in suicide after April 2017 only for the teenage group. How does this pattern of results support the argument that the show led to increased teen suicide? 

If you want to talk to someone or are experiencing suicidal thoughts, text the Crisis Text Line at 741-741 or call the National Suicide Prevention Lifeline at 1-800-273-8255. Here is a list of international resources. 

Who's more likely to claim expertise?

The journalist's article was illustrated with a photo of a businessman. Is that the best choice, given the study? Photo credit: Yuri Arcurs/Getty Images

Are you familiar with proper numbers? How about declarative fractions? 

If you claimed that you knew these concepts, you're probably fooling yourself....because declarative fractions and proper numbers are not real math things. 

A Washington Post journalist summarized a study on who is more likely to falsely claim expertise, or as some folks might put it: to "BS." The article's headline reads, "Rich guys are most likely to have no idea what they are talking about, study suggests." 

What did the study do?

Study participants were asked to assess their knowledge of 16 math topics on a five-point scale ranging from “never heard of it” to “know it well, understand the concept.” Crucially, three of those topics were complete fabrications: “proper numbers,” “subjunctive scaling” and “declarative fractions.” Those who said they were knowledgeable about the fictitious topics were categorized as BSers.

The sample for the study was international:

The study drew from the Program for International Student Assessment, which is administered to tens of thousands of 15-year-olds worldwide. The test included a background questionnaire that captures demographic information, along with students’ attitudes toward the subjects they study in school. That section of the test included the questions about math knowledge.

Here is the pattern of results for gender (You can see tables of the data in the journalist's article).  

The data revealed that boys across all nine countries were significantly more likely than girls to pretend expertise, with the difference between the two working out to nearly half a standard deviation in some countries — a big gap, statistically speaking.

And here's the description of the results for income:

There’s also a significant class-based difference, with respondents from the wealthiest households showing a greater proclivity toward overstatement than those from the poorest.  

a) Here's a statistical validity question. When the authors say that the boys are about a half a standard deviation higher than girls in some countries and report it's "a big gap, statistically speaking," they are referring to effect size. In fact, if two groups are half a standard deviation apart, the effect size is probably about d = 0.5. Is that considered a small, medium, or large effect according to Cohen's conventions? 

b) Here's an external validity question. What were the age and income levels of the students in the study? Reflect on the headline, which refers to "rich guys," and the photo accompanying the story, which shows an adult man. What do you think--what external validity issues are raised here? 

c) Here's a construct validity question. The construct the journalist claims is being measured here is the "tendency to BS." How was that construct operationalized in this study? What do you think of that operationalization? Can you think of other ways to operationalize it? 

d) You might think we're ready for an internal validity question. However, internal validity doesn't seem relevant here, because nobody seems to be making a causal claim....or are they? Could you make a causal claim here? 

e) Advanced question: You might wish to investigate the table of data in the article that depicts the "BS gaps between boys and girls." Do you think that this table really needed separate columns for boys, girls, and "gap"? Why was it formatted this way? How else might you present these data? 

f) At the end of the article, the scientists are quoted about the study's implications and limitations. One of the study's original authors says, 

The authors also point out that their study was narrowly defined and restricted to the realm of mathematics. “Ideally,” they wrote, “future research should try to include a greater number of fake constructs in order to maximize precision of the bulls--- scale.”

Which of the four big validities seems to be addressed in this quote? 

Human intelligence over time

When people knew they could look up facts online, they didn't remember them as well.

Here's an article on how intelligence might be changing over time, from Australia's Sydney Morning Herald. It's a great opportunity to practice classifying claims and identifying studies. 

One point the article makes is that humans have been getting more intelligent in general. For example:

For the last century, we’ve been getting smarter - fast. The average IQ of the population has been increasing by about three points per decade.

Even Mensa has noticed. “Our membership is growing at about 10 per cent a year,” says Teresa Wong, the organisation’s recruitment officer. “And about a third of them are under 18.” [...]

The same increase in IQs over the last century has happened in nearly every other developed nation. It’s known as the Flynn effect after James Flynn, the New Zealand researcher who discovered it.

There are several theories for why this has happened: we’re better educated, modern education systems mould our brain in ways IQ tests tend to like, and many of us will do cognitively-taxing things like play video games during our leisure hours.

a. What kind of claim is it to say that "the average IQ of the population has been increasing by about 3 points per decade"? (Frequency, association, or cause?)

 b. What kind of data would you need to support this claim?

The journalist then shifts the focus: 

But something strange and troubling appears to be happening. In several countries – Norway, Denmark, Finland, Britain and France – IQ scores that rose for so long now seem to be dropping, at a rate of about 2.4 points per decade in recent decades.

c. If you were to sketch the average IQ scores of these nations over several decades, what would the graph look like? 

The journalist asks, 

Could it be that our smartphones are making us dumber? We can look up any fact in an instant. We never need to remember facts or directions or friends’ phone numbers. Is that making us more stupid?

d. What kind of claim is the journalist making, above?

e. Below are three studies the journalist describes. Two are probably correlational and one is probably experimental. Which is which?  (and, what are the two variables in each study?):

i. Studies have linked smartphone use with a decreased ability to exert high levels of focus and poorer attention control – although video games seem linked to better multitasking abilities.

ii. In one influential study, volunteers were asked to type newly-learned trivia facts into a computer. Those who were told the computer would remember the facts for them performed much worse on a later memory-recall test. The researchers dubbed it the Google Effect.

iii. In another study, volunteers were given a series of questions that pushed the limits of their brain. Those who said they were heavy smartphone users tended to perform worse and be less analytical. 

f. What kind of data would it take to convince you that the Internet is responsible for the recent declines in IQ around the world? 

You can read one of the original "Google Effect" studies here. 

Does loud music make you eat bad food?

That sounds delicious, thank you! Photo credit: Nathan Motoyama/EyeEm/Getty Images

June is food month on the ERM blog! This story's about the headline, Loud music makes you eat bad food?!?!?Whaaaaaaatever!

QRock 100.7 was one of several news outlets that had fun describing this study for its readers. 

Let's find out what kind of study was conducted to test the claim. The Q100.7 journalist wrote:  

A new study found loud music makes us more likely to order unhealthy food when we’re dining out.  A new study in Sweden found loud music in restaurants makes us more likely to choose unhealthy menu options.  And we’re more likely to go with something healthy like a salad when the music ISN’T so loud.

Researchers went to a café and played music at different decibel levels to see how it affected what people ordered.  Either 55 decibels, which is like background chatter or the hum from a refrigerator . . . or 70 decibels, which is closer to a vacuum cleaner.

And when they cranked it to up 70, people were 20% more likely to order something unhealthy, like a burger and fries.

They did it over the course of several days and kept getting the same results.  So the study seems pretty legit.

a) OK, go: What seems to be the independent variable in this study? What were its levels? How was it operationalized?

b)  What seems to be the dependent variable? How was it operationalized? Think specifically about how they might have operationalized the concept "unhealthy."

c) Do you think this study counts as an experiment or a quasi-experiment? Explain your answer. 

d) This study can be called a "field study" or perhaps a "field experiment". Why?

e) To what extent can this study support the claim that loud music makes you eat bad food? Apply covariance, temporal precedence, and internal validity to your response.

f) If you were manipulating the loudness of the music for a study like this, how might you do so in order to ensure it was the music, and not other restaurant factors, were responsible for the increase in ordering "unhealthy" food?

g) The Q100.7 journalist argues that the study seems "pretty legit." What do you think the journalist meant by this phrase? 

h) The study on food and music volume are summarized in an open-access conference abstract, published here. You might be surprised to read, contrary to the journalist's report, that the field study was conducted on only two days--with one day at 50db and the other at 70 db. How does this change your thoughts about the study? 

g) Conference presentations are not quite the same as peer-reviewed journal publications. Take a moment (and use your PSYCinfo skills) to decide if the authors, Biswas, Lund, and Szocs, have published this work yet in a peer reviewed journal.  Why might journalists choose to cover a story that has only been presented at a conference instead of peer-reviewed? Is this a good practice in general? 

 

 

Does smoking marijuana cause car fatalities?

The study found an estimated 12% higher rate of fatal accidents after 4:20pm on April 20. Credit: Lars Hagberg/AFP/Getty Images

REPLICATION UPDATE (posted Feb 25, 2019) Another set of researchers wondered why the study below compared April 20 to April 13 and April 27, rather than comparing April 20 to every other day in April (or to every other day in the year). As it turns out, when you compare April 20 to every other day of the month or year (not just the two originally selected) April 20 does not actually stand out in terms of fatal crash risk. 

This new analysis recategorizes this post. It used to be tagged an example of a quasi-experiment. Now it is a good example of (possible) cherry-picking, and an example of replication efforts. The newer published study's abstract includes these details:

Between 1992 and 2016, ‘4/20’ was associated with an increase in the number of drivers involved in fatal crashes (IRR 1.12, 95% CI 0.97 to 1.28) relative to control days 1 week before and after, but not when compared with control days 1 and 2 weeks before and after (IRR 1.05, 95% CI 0.92 to 1.28) or all other days of the year (IRR 0.98, 95% CI 0.88 to 1.10). Across all years we found little evidence to distinguish excess drivers involved in fatal crashes on 4/20 from routine daily variations.

There is little evidence to suggest population-wide effects of the annual cannabis holiday on the number of drivers involved in fatal traffic crashes.

 

Original post:

Here's a study that took advantage of "4-20", an unofficial holiday which people celebrate by holding pot-smoking parties starting at 4:20pm. Here's how the quasi-experiment was described in a New York Times story: 

Researchers used 25 years of data on car crashes in the United States in which at least one person died. They compared the number of fatal accidents between 4:20 p.m. and midnight on April 20 each year with accidents during the same hours one week before and one week after that date.

a) What are the "independent" and dependent variables in this study? (And why did I put independent variable in quotes?)

Here's how the journalist described the results:

Before 4:20 p.m. there was no difference between the number of fatalities on April 20 and the number on the nearby dates. But from 4:20 p.m. to midnight, there was a 12 percent increased risk of a fatal car crash on April 20 compared with the control dates. 

b)  Of the four quasi experimental designs, which seems to be the best fit: Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design? 

c) Sketch a graph of the results described. 

d) The Times reported that "The increased risk was particularly large in drivers 20 and younger." Why might the researchers have included this detail?

e) The Times's headline read, "Marijuana Use Tied to Fatal Car Crashes". What kind of claim is this? (Frequency, Association, or Cause?)

f) To what extent can these results support a causal claim about marijuana causing crashes? Apply the three causal criteria to this design and results. 

 

 

 

Does legal marijuana lower opioid prescriptions? A quasi-experiment

Legalizing marijuana is associated with lower rates of opioid prescriptions in those U.S. states. Photo: Gina Kelly/Alamy Stock Photo

Opioid addition is a major health crisis in the United States.  Deaths from overdose increased dramatically in the last 5 years. Opioid addiction sometimes starts when a person in pain is prescribed legal opioid drugs by a physician. Opioid prescriptions can also be sold illegally. For these reasons, opioid prescription rates are an indicator of opioid abuse in a particular region.  

Some public health researchers have investigated whether legalizing marijuana can reduce rates of opioid use and abuse. Marijuana is an alternative for controlling chronic pain that, according to many experts, has a lower addiction risk. Recently, researchers published two studies, both with quasi-experimental designs, that tested whether legalized marijuana could lower the rates of opioid prescriptions. Like many quasi-experiments, the researchers took advantage of a real-world situation: Some U.S. states have legalized marijuana and other states have not.

ABC news covered the the research. There were two studies with similar designs, but we'll focus on the first one:

One looked at trends in opioid prescribing under Medicaid, which covers low-income adults, between 2011 and 2016. It compared the states where [medical] marijuana laws took effect versus states without such laws.... 

Results showed that laws that let people use marijuana to treat specific medical conditions were associated with about a 6 percent lower rate [over the years studied] of opioid prescribing for pain. That's about 39 fewer prescriptions per 1,000 people using Medicaid.

And when states with such a law went on to also allow recreational marijuana use by adults, there was an additional drop averaging about 6 percent.

Questions:

a) What is the "independent" variable in this quasi-experiment? What is the dependent variable? Was the independent variable independent groups or within groups? 

b) What makes this a quasi-independent variable? 

c) Of the four quasi experimental designs, which seems to be the best fit: Non-equivalent control group posttest only? Non-equivalent control group pretest-posttest? Interrupted time series design? Non-equivalent control group posttest-only design?

d) How might you graph the results described above? 

e) To what extent can these data support the causal claim that "legalizing marijuana, either for medical use or recreational use, can lower the rates of opioid prescriptions in the Medicaid system"?

 

The two studies are available open-access through JAMA Internal Medicine. Here's the study using Medicare Part D prescription rates, and here's the study using Medicaid prescription rates. 

 

Suggested answers

a) The independent variable was whether a state had legalized marijuana or not. It was independent groups (states either had, or had not, legalized the drug). The dependent variable was the number of opioid prescription rates through Medicaid. Another variable, somewhat difficult to discern from the journalist's description, was year of study (from 2011 to 2016)

b) This IV was not manipulated/controlled by the experimenter. The researcher did not decide which states could legalize marijuana or not.

c) This is probably best characterized as a non-equivalent control group, pretest-posttest design. There were two types of states (legalized and not) and one main outcome variable: opioid prescriptions. The prescription rate was compared over time (from 2011 to 2016), making it pretest-posttest.

d) Your y-axis should have "opioid prescriptions" and the x-axis should include the years 2011 to 2016.  You could then have "States with legalization" and "States without legalization" as two different colored lines. 

e) The results of the study show covariance (States with legalized marijuana had lower opioid prescriptions). The fact that they compared opioid prescriptions over time (2011 to 2016) suggest that the design is able to establish temporal precedence. Presumably (although this is not clear from the articles), 2011 represents a year before many of the marijuana laws took effect and 2016 data occurred after the laws had been active.
As for internal validity, it's possible that states that legalize are different in systematic ways than states that do not. For example, states that legalize marijuana are more likely to be in the North and West, have lower poverty rates, and so on. However, the pretest-posttest design, in which they studied the "drop in opioid prescriptions over time" rather than "overall rate of opioid prescriptions" helps minimize some of these concerns. As with most quasi-experiments, causation is not a slam-dunk, because the experimenter does not have full control over the independent variable. 

Are smartphones destroying a generation?

People in studies who spend more time socializing, rather than time on their phones, are generally happier. Photo: Maria Taglienti-Molinari/Getty Images

Are smartphones making young people lonely, anxious and depressed? Are teens spending time on phones instead of dating, driving, or drinking? That's the argument of a new data-based book by psychologist Jean Twenge.  

You can watch a video interview with Dr. Twenge on CNN here, complete with some of the graphs from Dr. Twenge's recent book on this topic, which reflects her research. There's also an article in The Atlantic here, which contains a condensed argument from the book. 

Several graphs presented in the CNN interview depict patterns of teenage behavior over time, from nationally representative surveys of youth. The graphs show how various healthy activities such as "hanging out with friends" dropped  starting in 2012. Twenge's explanation is that 2012 is the first year that more than 50% of Americans owned smartphones. Twenge argues that instead of going out with friends, driving, or being independent, today's teenagers are staying home and connecting with friends only via Snapchat and Instagram. 

a) The graph in the CNN story (see minute 1:50 to  2:05; and also here) shows several trends over time. These figures come from a study that is quasi-experimental. Which type of quasi-experiment does this seem to be?  

b) Take a look at the "More likely to feel lonely" figure, at minute 2:02. (again, you can also see it here, by scrolling down) The change in loneliness after 2007 has been described as "dramatic" and "precipitous". What do you think? Specifically, look at the y-axis of the graph. How dramatic would the data look if the axis ranged from, say, 0 to 100? 

c) Twenge's argument is that the advent of cell phones in 2012 is responsible for decreased social contact and increased loneliness, anxiety, and depression of youth. What might be some plausible alternative explanations for the pattern, other than cell phones? (that is, what are some internal validity threats?)

 

The data above are longitudinal data, collected over time. Twenge's research has also included correlational studies collected in one group of teenagers at a single point in time. This report in the Washington Post presents the results of an empirical study that found, among other things, that teens who spent more time doing Internet, texting, computer games, or social media were lower in happiness. 

 

d) Scroll down to the graph created by the Washington Post, which is titled "What makes teens happy?" You'll see that each bar represents a correlation between one use of time and teen happiness. What does a gray bar, or negative correlation mean (For example, what does the -0.11 correlation mean for Internet? Sketch a little scatterplot of this correlation. 

e) In the same graph, what does a blue bar, or positive correlation mean (For example, what does the 0.14 correlation mean for Sports or exercise?) Sketch a little scatterplot of this correlation. 

f) Select the correlation mentioned in d.Does this correlation, on its own, allow us to conclude that time on the Internet causes lower levels of happiness? What third variables might be responsible for this negative correlation? What about temporal precedence of the two variables? 

g) How might you describe the effect size of these correlations--are they weak, moderate or strong? How do you know?

h) Consider both the longitudinal data (in Questions a, b, and c) and the correlational data (in questions d-g). Both types of studies support the same conclusion, which make them an example of "Pattern and Parsimony." Explain why this is the case, in your own words. 

 

Instructors: You and your students might also be interested in a curvilinear relationship described in the same Post piece: 

The report’s findings were not all dire: Teenagers who get a small amount of exposure to screen time, between one and five hours a week, are happier than those who get none at all. The least happy ones were those who used screens for 20 or more hours a week.

 

 

 

 

 

 

 

Economics of sleep

The sun sets in Amarillo, TX an hour later than it does in Huntsville, AL though they are on the same time zone. Amarillo residents get less sleep and earn more money: Is there a causal connection? Photo: Creativeedits/Wikimedia Common

Sleep is an essential human function and getting more sleep is associated with improved mood, cognitive performance, and physical performance. Therefore, it might make sense that sleep would improve people's productivity and ability to earn money. That's the topic of a Freakonomics episode on the "Economics of Sleep." You can read the transcript or listen to the 45 minute episode here. (The section I focus on starts around minute 10.)   

Freakonomics' hosts interviewed a set of economists (including Matthew Gibson, Jeff Shrader, Dan Hamermesh, and Jeff Biddle) about their research on sleep, work hours, and income. The economists mentioned that, in order to establish a causal link between sleep and income:

What we need is something like an experiment for sleep. Almost as though we go out in the United States and force people to sleep different amounts and then watch what the outcome is on their wages.

While it is theoretically possible to conduct such an experiment, it is practically difficult to assign people to different sleep conditions for a long enough period of time to notice an impact on their wages. So the economists took an alternative path and used quasi-experimental data. In a creative twist, they compared wages at two ends of a single American time zone. The example they gave is Huntsville, AL and Amarillo, TX. Here's why. Gibson stated: 

It turns out that ever since we’ve put time zones into place, we’ve basically been running just that sort of giant experiment on everyone in America.

The story continued. You'll see the transcript version quoted below:

Consider two places like Huntsville, Alabama — which is near the eastern edge of the Central Time Zone — and Amarillo, Texas, near the western edge of the Central zone. [...]

...even though Amarillo and Huntsville share a time zone, the sun sets about an hour later in Amarillo, according to the clock, and since the two cities are at roughly the same latitude as well, they get roughly the same amount of daylight too.

So you’ve got two cities on either end of a time zone, roughly the same size — just under 200,000 people each — where, according to the clock time, sunset is an hour apart. Now, what good is that to a pair of economists interested in sleep research?

GIBSON: It turns out that the human body, our sleep cycle responds more strongly to the sun than it does to the clock. People who live in Huntsville and experience this earlier sunset go to bed earlier.

And the people of Amarillo go to bed quite a bit later. You can see this in data from the American Time Use Survey.

GIBSON: If we plot the average bedtime for people as a function of how far east they are within a time zone, we see this very nice, clean nice straight line with earlier bedtime for people at the more eastern location.

But since Huntsville and Amarillo are in the same time zone, people start work at roughly the same time, which means alarm clocks go off at roughly the same time.

GIBSON: That means if you go to bed earlier in Huntsville, you sleep longer.

The economists didn't use only Huntsville and Amarillo--they also conducted multiple comparisons of cities around the U.S. that were similarly on each end of a single time zone. Using "city of residence" as their quasi-experimental operationalization of "amount of sleep", the economists were ready to report the results for wages: 

So now Gibson and Shrader plugged in wage data for Huntsville vs. Amarillo and other pairs of cities that had a similar sleep gap.

GIBSON: We find that permanently increasing sleep by an hour per week for everybody in a city, increases the wages in that location by about 4.5 percent.

Four and a half percent — that’s a pretty good payout for just one extra hour of sleep per week. If you get an extra hour per night, Gibson and Shrader discovered — here, let me quote you their paper: “Our main result is that sleeping one extra hour per night on average increases wages by 16%, highlighting the importance of restedness to human productivity.”

Questions:

a) What is the independent variable in this time zone and wages study? What is the dependent variable?

b) Is the IV independent groups or within groups?

c) Which of the four quasi-experimental designs is this? Non-equivalent control group posttest only, Non-equivalent control group pretest-posttest, Interrupted time series, or Non-equivalent control group interrupted time series? 

d) The economists asserted, "sleeping one extra hour per night on average increases wages by 16%" (italics added). What do you think? Can their study support this claim? Apply the three causal rules, especially taking note of internal validity issues that this study might have. 

e) If you consider only one pair of cities, there are multiple alternative explanations, besides sleep, that can account for wage differences. Name two or three such threats (considering Huntsville and Amarillo as an example). Now consider, how might many of these internal validity threats be reduced by conducting the same analysis over many other city pairs?  

f) This Freakonomics episode was aired in 2015, but the study (about time zones) they reviewed is not yet published. What do you think about that? 

 

Answers to selected questions

a) The IV is "Hours of sleep" (but you could also call it "location on the time zone: East or West") and the DV is "Wages".

b) The IV is independent-groups.

c) Non-equivalent control group posttest only.

d & e) The results of the study support covariance: People in cities in the Eastern portion of time zones get more sleep and have higher wages than people in the Western portions. Temporal precedence is unclear, I think: Because the data were collected at the same time, it's not clear if the timezone came first, leading to more sleep and higher wages, or if people began to earn higher wages first, and then systematically moved Eastward. (However, the second direction certainly seems less plausible than the first.)

As for internal validity, if we consider only the city pair of Huntsville and Amarillo, we could come up with several alternative explanations. The two cities have different historical trajectories and different ethnic diversities; they are in two different states that have different fiscal policies and industry bases. Perhaps Amarillo has poorer wages in general and people are losing out on sleep there because they are working more than one job. However, these internal validity threats become less of an issue when you consider multiple pairs of cities. It is less plausible that internal validity threats that apply to one city pair would also, coincidentally, apply to all the other city pairs that are at opposite ends of a time zone. 

Even though the method is fairly strong, psychologists would be unlikely to make a strong causal claim simply from quasi-experimental data like these, because the independent variable is not truly manipulated. Nevertheless, the method and results of this quasi-experiment are certainly consistent with the argument that getting more sleep may be a factor in earning higher wages.