Everyday Research Methods

Fewer babies on Halloween?

This story reports that pregnant women are less likely to go into spontaneous labor on a negatively toned holiday (Halloween).  Similarly, they are more likely to do so on a positively toned holiday (Valentine's Day).

Here's a quote from the New York Times story:

In a study published this month, however, researchers at the Yale School of Public Health reviewed millions of birth certificates and found a 5.3 percent dip in spontaneous births on Halloween, suggesting that a mother’s mental state may play a role in when she goes into labor.

a.) Is this a frequency claim or an association claim?

b.) This research result might be seen as sort of a cute, public-interest story.  But it also seems to be testing a theory.  What is the theory, and how do the data support it?  

c.) The abstract for the empirical journal article that originally published this result provides a lot of information on the study.  According to the abstract, what was the comparison condition for the study? What was their sampling technique? 

Suggested answers

a.)  This is a tough call!  On the one hand, simply stating that there are fewer spontaneous births on Halloween is a frequency claim.  On the other hand, it would be an association claim if you stated that the positivity of a holiday (Halloween versus Valentine's Day) is associated with the number of spontaneous births. 

b.) According to the abstract, the theory is that health outcomes can be affected by cultural representations and stereotypes.  The data support the theory by using results from two holidays with distinct cultural representations.  Spontaneous births appear to fluctuate with the positivity of these two holidays, supporting the theory.

c.) In this study, the dependent variable is spontaneous birth rate.  The comparison conditions (the independent variable levels) were achieved by comparing birth rates on the target day (Halloween or Valentine's Day) with birth rates during the week before and after the target day. 

According to the abstract, the study appears to have conducted a census--they used all possible U.S. birth certificates for the selected date ranges. 

Testosterone levels in Dads

Journalists have been covering a recent empirical article published in the Proceedings of the National Academy of Sciences.

The study in question is a quasi-experiment; it tracked the testosterone levels of a probability sample of men in the Philippines. The main result was that compared to men of the same age who'd never had children, men who became fathers showed a steep drop in testosterone levels after the birth of a child. 

“The men with the highest testosterone levels were more likely to become fathers by the time we followed up with them five years later,” said Christopher Kuzawa, a study author and an associate professor of anthropology at Northwestern University, in an interview. “Once they became fathers, their testosterone plummeted, quite a bit, more than any of the other groups.” (Quote from Bloomberg)

 

a.  Can you explain what makes this a quasi-experiment?  What are the "independent" and dependent variables in these two findings?   What kind of design was used? 

b.  Sketch a graph the results that are described in the quote above. Does this combination of design and results control for internal validity threats?  Can you make a causal statement? 

Many popular press outlets covered this story.  This article in Slate magazine provides a critique--it lists the kinds of claims that journalists made about testosterone and parenting, compared to what the original empirical research showed. 

c.  Read the Slate article and reflect on the journal-to-journalism cycle.  Why do you think so many journalists moved beyond the data, according to the Slate author? 

d.  Use an Internet search to find other popular press coverage of this story.  What kind of claims are the journalists making about the study? 

e.  Because the journal, Proceedings of the National Academy of Sciences, is carried by many university libraries, you can look up the original paper and check the results for yourself.  Does the Slate article's author get the findings right in his own critique? 

Prenatal folic acid and child speech development

A journalist's headline has reported, "Folic acid tied to better toddler talk".

a. What kind of claim is this? Frequency, association, or cause?

Here is a summary of the main finding:

Women who took folic acid supplements in the first two months of pregnancy were less likely to have kids with severe language delays in a new study from Norway.

b. Given this summary, what are the variables in this study?

Here are some of the methodological details:

The researchers gave surveys to close to 40,000 Norwegian women a few months into their pregnancies. Those included questions on what supplements women were taking in the four weeks before they got pregnant through eight weeks after conception.

Then, when their kids were three years old, Susser and his colleagues asked the same women about kids' language skills, including how many words they could string together in a phrase.

c. The information above can help you interrogate construct validity. How well does it seem they measured their variables?

Finally, consider this comment:

The pattern remained after Susser's team took into account other factors that were linked to both folic acid supplementation and language skills, such as a mom's weight and education, and whether or not she was married.

d. If you've read Chapter 8, you can probably figure out what this means. Explain, in your own words, how the researchers can "take into account" these other factors.

Can makeup change the way women are perceived?

Recently, researchers tested how makeup affects people's impressions of women. Here is how the story was covered by ABC News--the story includes a video.

In the study, a set of female targets posed for photos in which they were wearing no makeup, some makeup, professional makeup, or full, "glamour" makeup. Then the photos were rated by a large number of observers. You can read about the details of the study on the ABC News website.

The results showed that for the most part, women wearing makeup were rated more positively on all four ratings. Here are some quotes from the lead researcher:

"We found that when faces were shown very quickly, all ratings went up with cosmetics in all different looks," said Nancy Etcoff.... "The women were judged as more competent, likable, attractive and trustworthy."

When the photos were shown more slowly, the results were a little bit different:

"When they got to the more dramatic makeup looks, people saw them as equally likable and much more attractive and competent, but less trustworthy," Etcoff said. "Dramatic makeup was no longer an advantage compared to when people saw the photos very quickly."

Here are some questions about the study:

a.) Consider only the study in which participants viewed the photos for 250 ms. What were the independent and dependent variables in this study? Was the independent variable between-subjects or within subjects?

b.) Sketch a graph of the results of the 250ms study. What is the best way to graph these results?

c.) Is this study an experiment? Does it support the causal claim in the headline, "Makeup makes women seem more competent"?

d. ) Why did the authors conduct the study under two conditions--250 ms of judgment as well as a slower one? (as you reread the article, look for the part about "cognitive pressure".) What do these conditions suggest about the generalizability and real-world applicability of their findings?

Suggested answers

a.) The independent variable was the degree of makeup that appeared on each target. It had four levels. According to the description in the web story, the independent variable was probably within-subjects--participants viewed all four levels of makeup (none, some, professional, and "glamour"). The dependent variables were the outsiders' ratings of likability, competence, attractiveness, and trustworthiness.

b.) You would probably make a bar graph with "level of makeup" on the x-axis and "rating" on the y-axis. You might have presented all four DV's on the same graph, labeling four different colored bars or lines that represent likability, competence, attractiveness, and trustworthiness.  (To see the actual results of the study, of course, you'd need to look at the original research.)

c.) This study is an experiment--it manipulated the independent variable (makeup level) and measured the dependent variables. Because it was an experiment, the study meets the temporal precedence rule. The results of the study also show covariance--women wearing makeup were judged more favorably. What about internal validity? By using each target as her own control (and only varying how much makeup each was wearing), it helps ensure that makeup use was not confounded with the attractiveness of the women. That shows good internal validity. In addition, the researchers did not let the women in the photos know how much makeup they were wearing, preventing the target women from inadvertently posing more confident facial expressions. Because the experiment appears to be well done (and meets all three causal rules) we can conclude that makeup can cause women to appear more competent. 

d.) The researchers seem to be using the 250ms version of the study as a way of mimicking real-world situations in which people make quick decisions or when they are distracted. The basic pattern of results replicated across two studies (with the exception of the trustworthiness ratings of women in the "glamour" condition), supporting the study's replicability and external validity. In addition, by presenting 25 diverse women as targets in this study, the researchers are able to say that the makeup effect can probably generalize to a variety of female targets. 

Tracking happiness on Twitter

A recent study used Twitter text to track people's moods throughout the day.
Here's how the study was covered by wired.com.

The journalist explained that:

Using Linguistic Inquiry and Word Count, a text analysis program that quantifies the emotional content of statements, Golder and co-author Michael Macy analyzed a total of 509 million tweets generated over two years by 2.4 million people in 84 countries.

The resulting trends — positive moods starting high in the morning and declining through the day, peaking overall on weekends — held steady around the world.

a) One thing that is novel about this research is the unique operationalization of their main variable, happiness. What is their operationalization, and why do the scientists believe it is a good measure of happiness?

b) Here is another quote from the Wired article:

Many studies of how moods — or, more technically, positive and negative affect — change from minute to minute and day to day rely on self-reported surveys, which can be inconsistent if not misleading. The subjects of these studies also tend to be undergraduate students from western colleges, a group that’s not always representative of humanity at large.

Twitter users, of course, don’t represent humanity either. But the culture- and globe-spanning size of the software platform’s community, and their constant generation of data that can be cross-referenced and correlated and otherwise computationally investigated, make them alluring to researchers.

When he writes, "Twitter users, of course, don't represent humanity either," which validity is the author addressing? What evaluation does the author make of that validity here?

c) The original research appeared the journal Science, a widely respected journal that your university library probably subscribes to. Use your library's tools to find the original article. What aspects of the original article did the journalist leave out?

d) This same study was featured by a number of websites and newspapers. Use an Internet search to find out how other journalists covered the same scientific study. Which news story do you think describes the original study most accurately, completely, and clearly?

e) The study's construct and external validities both seem strong. What does that mean to you--what conclusion can you make now?