Everyday Research Methods

Psychological research: How to talk with family you disagree with

The journalist provides a mixture of empirical evidence and authority to support advice for how to talk to family you disagree with.
  Photo: PeopleImages.com - Yuri A/Shutterstock

During the holidays, we may find ourselves interacting with family members who don't agree with us--especially on politics.

Here's a story from NPR news with advice on how to navigate these situations. It provides practice in identifying variables, classifying claims, and evaluating whether a claim is supported by empirical evidence or not. 

Political conversations with family members might fill us with dread, especially in light of this claim reported in the story:

Polling data from SNF Agora Institute at Johns Hopkins University shows that almost half of the U.S. electorate thinks members of the opposing political party are "downright evil." In a 2022 Pew Research Center study, growing numbers of Americans said members of the other party are dishonest, immoral and closed-minded.

a) In the quoted text above, what type of claim is being made--frequency, association, or cause?

b) In the quoted text above, what is/are the variable(s) in the claim?

c) In the quoted text above, what kind of evidence does the journalist provide--experience, authority, or empirical research? (You can review these terms in Chapter 2.)

 

Here's another excerpt from the story:

Neuroscience has shown that when two people agree, their brain activity is more synchronized than when they disagree.

Yale School of Medicine neuroscientist Joy Hirsch led a 2021 study that found that people's brains lit up in similar ways when they agreed. She thinks this means those people share more information and that the two people are more in consensus. Versus when people disagree: Their brains act like a cacophony instead of a harmonious duet.

d) In the text from the quote above ("Neuroscience has shown..."), what type of claim is being made--frequency, association, or cause?

e) In the text above, what are the variables in the claim?

f) In the quoted text above, what kind of evidence does the journalist provide--experience, authority, or empirical research? 

 

Here's another section: 

Tool 1: Focus on your breathing 

When we face potential conflict, our body's automatic response may not help us regulate our emotions. [...] "We get angry because we hear someone say something that we feel is just so wrong and we have to tell them that it's wrong," says Ken Barish, a psychologist at Weill Cornell Medical College who wrote the forthcoming book Bridging Our Political Divide. "So we have this mixture of anxiety and anger, and over time that becomes resentment. And contempt — and contempt is a very destructive interpersonal process."

But Briscoe-Smith says we can begin to move through these emotions by slowing down and refocusing on breathing. This can help combat the body's automatic response to conflict, helping us think more clearly and move to the next step.

g) Advice is usually a causal claim, because it suggests that if you do ____, it will cause ___.   In the text above, what are the variables in the causal claim? 

h) In the quoted text above, what kind of evidence does the journalist provide--experience, authority, or empirical research? 

 

And here's one more element you can analyze:

Tool 3: Empathy 

Humanize the other person you're talking to by asking about their lives, their families, their hobbies — not just their opinion on a single topic. That can help create more common ground outside of the conversation at hand.

i) As before, advice is usually a causal claim, because it suggests that if you do ____, it will cause ___.   In the text above, what are the variables in the causal claim? 

j) In the quoted text above, what kind of evidence does the journalist provide--experience, authority, or empirical research? 

 

Reflection

k) Looking back on questions h) and j), What do you think about this journalist's use of authority over empirical evidence? And, what kind of empirical evidence might you like to see to support their advice? 

 

Can stress turn your hair gray?

Correlational and experimental research seems to converge on the conclusion that stress causes gray hair. Photo: Kunlathida6242/Shutterstock

Can stress turn your hair gray? Here's a story on gray hair research, written by a New York Times science journalist. In our culture, we certainly believe that gray hair can be caused (in some cases at least) by stress. There is clear evidence that gray hair is inherited genetically, apparently stress plays a role, too 

Here's how the journalist introduces the first study:

In past studies, researchers have asked participants to fill out questionnaires about their hair color and stress levels, and then the scientists would see if they could link them.

In one study published in 2016, for instance, scientists surveyed more than 1,100 young Turkish adults and found that the 315 who reported prematurely graying hair had higher stress levels than those who didn’t. 

a) Identify the two variables in this study.

b) Were the variables manipulated or measured?

c) Was this a correlational or experimental study? 

d) Can the results support a claim that stress causes premature gray hair? Why or why not? Apply the three criteria for causation: Covariance, Temporal Precedence, and Internal Validity.

For question d above, you should have responded no; although this study did show a relationship between premature graying and perceived stress, the study doesn't establish temporal precedence since both variables were measured at the same time. And, it's possible some third variable is responsible--for example, maybe older participants have both more stress and more gray hair. (Note that even though genetics is the biggest predictor of premature gray hair, genetics isn't a confound here because the genes for gray hair don't also predict perceived stress.)

To show causation, it helps to have an experiment. Here's an example: 

But a mouse study published in 2020 took the research a step forward. In it, researchers stressed mice in various ways, including by injecting them with a chili-pepper-like chemical that induced a “fight-or-flight” response. This caused them to release the stress hormone norepinephrine, which, in turn, depleted their hair follicles of the stem cells involved with adding pigment to mouse fur. The hair then grew in gray.

But studies on this topic are challenging to perform on people, because researchers can’t ethically induce artificially high stress responses in humans like they can in animals or cells, Dr. Hsu said.

e) What is the independent variable in this study?

f) What were the dependent variables (there were two)? 

g) If you've read Chapter 9, you'll recognize this as a mediator argument: "[The chili-pepper chemical] caused them to release the stress hormone norepinephrine, which, in turn, depleted their hair follicles of the stem cells involved with adding pigment to mouse fur. The hair then grew in gray." Can you draw this mediator argument using boxes for each variable? 

Here's one more study on humans that was better able to manage temporal precedence: 

One small human study published in 2021 still advanced the narrative: Researchers plucked various strands of hair from 14 volunteers who had at least some graying. Several of the strands were fully gray, some were partially gray and some hadn’t grayed at all. The scientists then created high-resolution digital images of the hairs and calculated when each strand went gray using estimates of how quickly hair grows.

They also asked the participants to plot out stressful experiences from the past year on a timeline, and rank them from least to most distressing. The researchers found that when a strand turned gray frequently corresponded with the most stressful moment of that volunteer’s previous year.

h) There seem to be two variables in this study. Can you name them and their levels?

i) Was this a correlational or experimental study? 

Taken together, the three studies here can be seen as an example of "pattern and parsimony." Some of the studies are correlational, others are experimental. Some are on humans and some are on animals. But they all seem to parsimoniously support an argument that stress precedes gray hair. What do you think?

 

Really? Prenatal cannabis exposure and early language development

Photo: Viacheslav Lopatin/Shutterstock

A story in PsyPost was headlined with, Prenatal cannabinoid exposure appears to have a strange impact on early language development. 

You might have noticed that the verb phrase, "has a strange impact on" is a causal one. We are being asked to believe that prenatal exposure to cannabis causes something in children's early language development.  I hope your causality suspicions are activated here, since it's almost certain that both cannabis exposure and language development are measured variables. This was probably a correlational study. 

Let's read on to find out what the mysterious "strange impact" is.

The journalist reported on a study in the open-source journal, Frontiers in Pediatrics. The PsyPost journalist drops the "strange impact" in the first paragraph:

...infants exposed to cannabis in the womb tend to exhibit improved language development by their first birthday. This surprising finding sheds light on the impacts of prenatal cannabis exposure on one aspect of early childhood neurodevelopment. However, experts advise that women avoid cannabis use during pregnancy due to known risks and unknown long-term effects.

Given past research, it's a bit surprising to find a positive relationship between cannabis use and child development, so that was the "strange impact" PsyPost mentions in their headline.

Here's how the journalist summarized the study's methods:

The researchers recruited 207 pregnant individuals and their 12-month-old infants who were part of The Safe Passage Study. This was a large prospective study that followed participants from 2007 to 2015.

Talavera-Barber and colleagues assessed the infants according to the Mullen Scale of Early Learning, a tool that assesses cognitive development in children from 2 days old to 68 months. 

[...]Prenatal cannabis exposure was collected from the mothers through self-report. Participants were categorized based on whether the exposure to cannabis occurred early (first trimester only; 51 participants) or late (second or third trimester; 18 participants), and they were randomly matched with unexposed participants for comparison (138 participants).

Don't be misled by the term "randomly matched with unexposed participants." A peek at the full empirical article shows that these were simply people in the sample who did not use cannabis. They were randomly selected from a larger set of participants in the study--but that doesn't mean they were "randomly matched." (In fact, I'm not sure I know what the journalist meant by "randomly matched" here.)

Questions

a) Based on what you've read so far above, complete this table of the variables in the study:

Variable name (stated at the conceptual level)	How was this variable operationalized?	Is this a measured or a manipulated variable?

Some of the results were summarized this way by PsyPost:

Unexpectedly, infants who were exposed to cannabis later in pregnancy scored higher in both expressive and receptive language areas than those who were not exposed.

Additionally, infants exposed early in pregnancy showed better gross motor skills, although there was no difference in fine motor skills and visual reception skills.

b) Sketch a bar graph of the relationship between exposure to cannabis later in pregnancy and expressive language scores.

c) Now apply the three criteria for causation. Sketch three diagrams following the model in Chapter 3, assigning one variable (cannabis exposure) to A and another (expressive language) to B. 

d) For the internal validity step, think of a specific third variable, C, that goes both with high levels of cananbis exposure and with higher levels of language development. 

e) Notably, if you read the original empirical article, you can see that the researchers used multiple regression to control for some "C" variables (i.e., third variables). They controlled for age, race, education, household income, prenatal alcohol and tobacco use, parent's anxiety, and infant sex. That means if you used one of these variables as your "C" variable in question d, we can probably rule it out. Can you think of an additional "C" variable--one that they did not control for and that is plausibly associated with both cannabis use and higher language development? 

Really? "Cutting out meat might help prevent snoring"

Does the study support the causal claim the journalist attached to it? Photo: Dusan Petkovic/Shutterstock

Guest post by Carli Fine, University of Delaware

A recent news article for the general public was headlined, “Cutting Out Meat Might Help Prevent
Snoring: Study”. The journalistic source was Health Day: News for Healthier Living, which makes recommendations for the general public and has news for medical professionals. The original empirical journal article  (open access) was entitled “Plant-based and vegetarian diets are associated with reduced obstructive sleep apnoea risk”.

This study offers a great opportunity to discuss how association claims differ from causal claims and how empirical journals differ from popular journalism.

In the empirical study (Melaku et al., 2024), researchers investigated the variables of adherence level to a general plant-based diet (PDI) and risk level for obstructive sleep apnoea (OSA). PDI was measured with a 24-hour recall method, as participants reported the foods that they had
consumed in the past day.  OSA was measured using a tool called the STOP-BANG (which combines data about people's Snoring, Tiredness, Observed snorting, Pressure (blood pressure), Body mass index (BMI), Age, Neck size, and Gender). The researchers found a negative relationship between PDI adherence and OSA risk (STOP-BANG scores), such that higher PDI adherence went with lower OSA risk.

a) The journalist's headline reads, "Cutting Out Meat Might Help Prevent
 Snoring". What's the verb in this headline? To what category of claim does this verb belong: Frequency, association, or cause?  (Use Table 3.4 below for a hint.)

Table 3.4

b) What were the study's variables?  

Name of the variable (Conceptual level)	What are the variable's possible levels?	Is this manipulated or measured?

c) How do you know this was a correlational study (and not an experiment)?

d)  Sketch a well-labeled graph of the result they describe. It’s okay if you don’t know the exact
data for each group. Before you start: Will your graph be a bar graph or a scatterplot? 

 

e) Does the study support a causal claim/headline? Name, and then apply, each of the three causal criteria to this situation. When you get to the third one, you should specify a plausible third variable that could explain the relationship.

Name of causal criterion	Application to this example

f) At the end of the popular article you'll see a “What This Means For You” box, copied below. Comment on this takeaway: Does the study support the claim in the box? 

g) Scholarly sources are sometimes referred to as primary source literature, while journalistic sources may be referred to as secondary source literature. Fill out the table below identifying who the authors are for each type of article. Which type of source should you use for a literature review?  You'll find Table 2.3 helpful. 

Type of Source	“Degree” of Source	Authors of Source	Intended Audience of Source	Examples of this Type of Source
Scholarly	Primary
Journalistic	Secondary

h) Journalistic sources tend to analyze, interpret, or summarize the content of primary source literature. Besides the headline, here are a few more examples of quotes that differ between the two articles. What are some of the differences in meaning? Why do you think that the journalist made these choices?

Journalistic Source	Scholarly Source
“People with diets highest in plant-based foods were 19% less likely to have sleep apnea compared to those with the lowest amounts of plant-based foods in their diet.”	“Participants in the fifth (highest) quintile of PDI had 19% lower odds (OR=0.81; 95% CI: 0.66–1.00) of sleep apnea risk compared to those in the first quintile (p-value for trend=0.008).”
“A plant-based diet worked more powerfully to lower sleep apnea risk in men”	“Overall PDI is primarily associated with reduced odds of OSA risk in males, not females.”

 

 

 

 

Suggested Answers:

a) The popular article title is “Cutting Out Meat Might Help Prevent Snoring: Study”. The verb is “prevent”, which is a causal claim verb. The empirical article title is “Plant-based and vegetarian diets are associated with reduced obstructive sleep apnoea risk”. The verb is “associated with” which is an association claim verb.

b)

Name of the variable (Conceptual level)	What are the possible levels of this variable?	Is this variable measured or manipulated?
Level of Adherence to a General/Healthy Plant-Based Diet (PDI)	Low to High	Measured
Level of Risk for Sleep Apnoea (OSA)	Low to High	Measured

c) This was a correlational study because all the variables were measured. In an experiment, at least one variable is manipulated.

d)

e)

Name of causal criterion	Application to this example
Covariance	The results show that level of adherence to a general/healthy plant-based diet was associated with level of sleep apnoea. Yes, covariance is achieved!
Temporal Precedence	This one is tricky! The researchers asked about both variables in the same questionnaire, at the same time. However, PDI was assessed based on foods consumed in the past 24 hours, which therefore took place before the STOP-BANG tool was administered. Yes, temporal precedence is achieved!
Internal Validity	This is a correlational study, so we can’t rule out any third variables that might be responsible for the underlying relationship. For example, people who adhere more to a plant-based diet may also choose to engage more in exercise and healthy behaviors that reduce risk for sleep apnoea. Alternatively, people who have a higher socioeconomic status (SES) might have better access to and means to afford fresh fruits and vegetables and also have better access to healthcare services leading to better overall sleep hygiene. No, internal validity Is not achieved!

f) The key takeaway uses the verb “reduce” which is a causal claim. The causal claim made by the popular article cannot be supported because the study did not manipulate a variable to establish internal validity. In fact, the researchers even write, “The cross-sectional nature of the study prevents us from inferring causality between dietary indices and OSA risk.”

g) 

Type of Source	“Degree” of Source	Authors of Source	Intended Audience of Source	Examples of this Type of Source
Scholarly	Primary	Researchers who conducted a scientific study	Other scientists, researchers, readers of this blog and students who have taken research methods!	European Respiratory Society, Nature, Science, Cell, Journal of the American Medical Association
Journalistic	Secondary	Journalists or reporters, typically employed by a news organization	General population	Health Day: News for Healthier Living, The New York Times, The Washington Post, BBC News, CNN

g) You should only use empirical/scholarly/primary sources for a literature review.

h) Journalists write for a general audience that might not have specialized knowledge in a particular field. They might oversimplify the research by avoiding technical jargon and statistical terms to make it less precise but more accessible. Journalists typically want their stories to be impactful for people’s daily lives. They might make causal interpretations of correlational studies to be more actionable.

Most American adults feel awkward

The journalist's uncritical acceptance of this frequency
claim's data is also awkward.
Photo credit:JRP Studio/Shutterstock

Here's an interesting set of frequency claims about what percentage of U.S. adults feel awkward. The data were presented at the content farm StudyFinds, under the headline, "That's awkward! 68% still often feel as insecure as they did as teens" 

Here's a snippet from the journalist's summary:

A survey of 2,000 U.S. respondents ages 25 to 45 found that as teens, people were most self-conscious about their body shape (65%), hairstyle (61%), and smile (61%). Many continue to feel this way, despite the average person feeling at their most awkward at age 17.

As adults, over half are still uncomfortable about their body shape (55%), hairstyle (53%), and smile (52%). More men than women are currently self-conscious about their glasses/contact lenses (51% vs. 39%), height (54% vs. 42%), hairstyle (57% vs. 48%), and smile (55% vs. 49%).

You might be wondering why a researcher might collect such specific data about what people feel awkward about. Turns out this study was  "Conducted by OnePoll on behalf of Smile Express, a doctor-monitored at-home aligner treatment for adults"--in other words, the poll was commissioned by a private company that sells ways people can improve their appearance. 

Questions

a) How might the commercial nature of this survey affect how the study was conducted? For example, what results might Smile Express be hoping to find? How might that affect the question writing strategy and sampling strategy? 

b) Which of the four big validities should we prioritize when reading about a frequency claim such as "over half are still uncomfortable about their... smile (52%)"? 

c) To help you consider external validity, here's some information about the study's sample. Based on this quote, do you know anything about the representativeness of the study's sample? 

This random double-opt-in survey of 2,000 U.S. adults ages 25–45 was commissioned by Smile Express between Aug. 7 and Aug. 10, 2023. It was conducted by market research company OnePoll, whose team members are members of the Market Research Society and have corporate membership to the American Association for Public Opinion Research (AAPOR) and the European Society for Opinion and Marketing Research (ESOMAR).

Discussion of question c:

To understand the paragraph above, you need to know what a "double-opt-in" survey is. As the name implies, an "opt-in" survey is one people volunteer for. For example, people might volunteer to participate in a study in exchange for cash, gift cards, or frequent flyer miles. A double-opt-in study is one in which people opt in twice; first they volunteer to be contacted by the commercial polling company, and then they opt in to a particular survey or poll. 

To test the effectiveness of opt-in polls, Pew Research once compared three commercial opt-in polls to three Interent panel samples which had used random sampling of addresses. They found that the opt-in polls resulted in larger errors compared to known benchmarks. For example, people in the opt-in polls were more likely to be receiving social security benefits than the general population and were less likely to have served in the military. In other words, opt-in surveys are not particularly representative, especially when compared to Internet panels, which use probability samples.

Given that opt-in polls are not particularly representative, why, then, might the StudyFinds journalist have described this sample as "random"? Maybe she misinterpreted the study's sampling strategy?

Advanced students will enjoy critically reading the website of the market research company OnePoll and identifying ways that it describes its research (its FAQ includes the question, "How do you ensure that data is statistically significant"?)

Cognitive profiles of risky drinkers

Risky drinkers were more sensitive to alcohol-related rewards and punishments. Photo:PanuShot/Shutterstock

Researcher Elizabeth Goldfarb's research on the cognitive styles of risky drinkers was the topic of a recent episode of the podcasts, Under the Cortext, produced by APS. You can listen to the 15- minute discussion between Dr. Goldfarb and Dr. Özge G. Fischer Baum on the podcast, or you can read the transcript of the discussion here.  The podcast does a great job of describing risky drinking and explaining some of the downsides of risky drinking.

What is risky drinking? Dr. Goldfarb describes it as follows:

 ...risky drinking is basically engaging in patterns of alcohol intake that are associated with higher risk for negative outcomes, both for physical well-being and for mental health. [For example] regularly engaging in binges where you consume a large amount of alcohol in a short amount of time, or just persistently having overall high levels of intake throughout your week.

The cognitive processes that Dr. Goldfarb studies are generalization and overgeneralization. Generalization is a type of learning: 

...say a dog bites you. You want to use that to inform how you interact with dogs in the future, right? You don’t want to just walk up to another dog, stick your hand in front of its nose, and I wonder what’s going to happen now. You want to be able to use your past knowledge, right? But you don’t want to take that to an extreme and start being really afraid of even seeing any dog far away from you across the street. So we’d think about the first one of just taking your knowledge and using it adaptively as generalization. But taking it too far would be a form of overgeneralization.

Goldfarb's team used a card game to study overgeneralization in both risky and lighter drinkers. They found that, compared to lighter drinkers, risky drinkers tend to overgeneralize associations with alcohol.  To find out how they operationalized "overgeneralization," see if you can follow their study paradigm based on her description: 

... we told people that they were going to play a card game to earn tokens that would add to a financial bonus they’d get at the end of the study. So first they learned which cards would earn them the tokens, and then they got to choose in the second phase, which cards did they want to play with? What was happening under the hood in this game was that one card was paired with tokens that were actually pictures of alcohol, and the other card was paired with tokens that were just pictures of common objects. And the reason we did that was to see, are they really focused on alcohol related reward, or is it just any kind of reward that they’re going to generalize more? And then the third card was very rarely paired with any kind of token.

[we found that]...people who engaged in riskier patterns of drinking behavior stretched these alcohol associations specifically out farther. So [later in the game] they showed this tendency [to overgeneralize--they wanted to use] cards even if they didn’t look very much like the one that was paired with alcohol, they still want to approach it, they still want to play with it.

Next, the research team dug deeper; they wanted to test if the overgeneralization effect might also occur for punishment learning:

...we did the study again, but now they were penalty tokens. They were going to take away money from our participants. And [risky drinkers] also over generalized alcohol punishment. They showed a tendency here to avoid cards that looked anything like the one that was initially paired with an alcohol punishment. [This pair of studies is] showing that there’s just this overall tendency to take what you learn about alcohol and spread it out to overgeneralize alcohol associations in a way that’s really specific to alcohol. But valence general, it doesn’t matter if it’s gain or loss.

You might be asking, why do we care how a risky drinker plays a card game? The podcast host asked that question, too, and Goldfarb replied:

...we know that memory plays a huge role in substance use. You can’t relapse in a place where you used to use if you don’t remember that you ever used there before, but we don’t know what it is that they’re remembering. So many of the projects we’re doing in the lab are trying to isolate. We have all these different memory systems, all these different types of representations, which are the ones that are really motivating this problematic behavior. And once we understand which memories they are, we can start to think about how can we target them, how can we help people form different, perhaps more protective associations

This is a challenging study to work with, but keep in mind that at the heart of every study are its variables. 

a) Classify the variables in the first study (reinforcement learning). 

Variable name	How was this variable operationalized?	What are the variable's possible levels?	Is this manipulated or measured?

b) Which kind of study is this? Correlational or experimental? 

c) See if you can sketch a scatterplot, bar graph, or line graph of the results they described. Later you can compare your sketch to the results in the empirical article (Figure 2, especially panel b). 

The empirical article reporting this study appears in Psychological Science; it contains images of the cards used and full descriptions of the results.

 

"Want to raise kind, generous kids? Take them to an art museum"

Did the study manipulate exposure to art museums?
Photo: AChubykin/Deposit Photos

On my first read of this headline, I was skeptical. "Probably a correlational study behind this one," I thought. As it turns out we are actually looking at an experiment. However, there's another problem with the journalist's coverage of this study. Read on.

The journalist writes for Inc.com, which  summarizes research for the business world. The headline was "Want to raise kind, generous kids? Take them to an art museum" The original empirical article appeared in Psychological Science. Let's work through it. The introduction starts out:

Everyone wants to raise successful kids, but a big part of being successful is being a kind and generous person. So how can parents give their kids the best while also doing their best to ensure they raise decent, thoughtful little people? 

They introduce the idea that art might make kids more generous: 

You may be wondering what possible connection a day viewing Van Gogh or Picasso could have to raising an empathetic kid. Certainly, museums make children more worldly and cultured, but how do they make them kinder? The link, according to the new study, recently published in Psychological Science, is awe. 

a) Before you read more, make a prediction. Given this introduction, what independent variable do you think the researchers probably manipulated? What variable do you think was measured?

Now let's read the details. There were two studies. In the first one, 

..the research team asked 159 volunteers aged 8 to 13 to watch short movie clips. Some of these clips were neutral, others cheerful, and others awe-inspiring. . 

In the second study, about 350 kids participated. The researchers used one of the same three clips for the awe variable (neutral, joy, and awe), but in this case, the generosity variable was operationalized differently:

The researchers then asked the kids [in Study 1] to determine how many items in a list of foods should be donated to a food drive for needy families. Alternately [in Study 2], the kids were given the option of donating their reward for participating in the study--a ticket to a local art museum--to a refugee family. 

(By the way, the original article reports that "The awe-eliciting clip was from the film Song of the Sea, in which the main character turns into a seal and transforms the city while flying above it.")

Back to the Inc.com story for the results: 

"Children who watched the awe-inspiring video chose to count 50 percent more items for the food drive than children who watched the joy-inspiring clip and more than twice as many items as children who watched the neutral clip. [in Study 2], children in the awe-inspiring condition were also 2 to 3 times more likely to donate their museum tickets than children in the joyful or neutral conditions," reports the Association for Psychological Science blog. 

b) Take a moment to classify the two variables in Study 1:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

c) Which kind of experiment is Study 1: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

d) And now for Study 2:

Variable name	What are the variable's possible levels?	Is this manipulated or measured?	Is this an IV or a DV?	For IVs: is it within groups or independent groups?

e) Which kind of experiment is Study 2: Posttest only? Pretest/posttest? Concurrent measures? Repeated measures? 

Look back at the description of the study. Did either Study 1 or Study 2 manipulate "taking kids to an art museum"? Nope. In fact, Study 1 took place online (kids watched video clips on their own computers). And Study 2 took place in a science museum. Neither of the studies manipulated art museum exposure. Art museum tickets were used in Study 2, but only as a way of operationalizing generosity.

That factual error didn't stop the journalist from asserting:

Parents ... can always go to their local art museum to give their kids a guaranteed dose of awe. 

Guaranteed dose of awe? That conclusion does not really follow from this study!  So here are a couple more questions. 

f) What could be an alternative headline for the Inc.com story--one that is more accurate to what the study actually manipulated?

g) The journalist appears to have generalized from one setting (watching video clips) to another (going to an art museum). Which of the four big validities is the journalist treading on here? 

h) Here's a construct validity question: The two studies operationalized the dependent variable, "generosity" in two different ways. Were each of them a good way to measure this variable? Why or why not? What are the strengths of operationalizing the variable more than one way?

i) Now for a question related to internal validity: The main independent variable was the content of the film clip the children saw. To ensure internal validity, what should the researchers hold constant about the film clips used in the three conditions? 

 

 

 

Really? "Sleep this way to add 5 years to your life"

A friendly reminder that when all the variables are measured, causal claims aren't supportable. Photo credit: Sleeping: MicrostockAsia/Deposit Photos

CNN's headline was "sleep this way to add 5 years to your life." This headline should immediately kick in your "correlation is not causation" spidey-sense.

a) What makes this headline causal?

b) What are the two variables in the headline? 

c) Why might you suspect this headline is based on a correlational study (rather than an experiment)?

Now that these preliminaries are out of the way, let's find out if the study behind the headline  can support causation.

Here's the splashy intro:

Want to live longer? Then prioritize sleep in your life: Following five good sleep habits added nearly five years to a man's life expectancy and almost 2.5 years to a woman's life, a new study found.

The article then provides advice such as getting at least 7 hours of sleep, and so on. The journalist links that advice to the following research summary:

"Recent studies have shown irregularity in sleep timing and duration have been linked to metabolic abnormalities ...," he said. "Encouraging maintenance of regular sleep schedules with consistent sleep durations may be an important part of lifestyle recommendations for the prevention of heart disease."

d) In the example above--What are the two variables in the study? What makes the study correlational? 

e) Apply the three causal criteria to the example in d). The study can support covariance (why?). What about temporal precedence? What about internal validity? What specific third variables might be responsible for the link between sleep timing and metabolic abnormalities?

 

Later in the piece, CNN shares data from another study, this one using multiple regression:

The preliminary study, presented Thursday at an annual meeting of the American College of Cardiology, analyzed data from over 172,000 people who answered sleep questionnaires between 2013 and 2018 as part of the National Health Interview Survey. The annual survey is done by the CDC and the National Center for Health Statistics.

 

Each of the five healthy sleep habits — falling asleep easily, staying asleep, getting seven to eight hours of zzz's, waking up rested and foregoing sleep meds — was assigned a number. People were scored on how many of the five habits they had.

 

About four years later, researchers compared those scores with National Death Index records to see if their sleep behaviors contributed to an early death from certain diseases or any cause.

 

The team then factored out other potential causes for a higher risk of dying, such as alcohol consumption, lower socioeconomic status and existing medical conditions.

 

f) Make a table of the variables in this study:

Variable name	How was this variable operationalized?	What are the variable's possible levels?	Is this manipulated or measured?
			(add rows as needed)

 

g) The study directly above meets both covariance and temporal precedence....can you explain why? 

h) Imagine that a critic says, about the study directly above, "Hmmm....it could be that healthy sleep habits predicted early death because of alcohol consumption--maybe people who drink more alcohol are more likely to have unhealthy sleep, and they are also more likely to die early." What do you say? 

 

Finally, if you're looking for an example of a moderator, we've got you:

 

Men who followed all five of the healthy sleep habits had a life expectancy that was 4.7 years greater than people who had none or only one of the five elements of low-risk sleep, the study found.

 

The impact of healthy sleep habits was much lower for women: Those who followed all five sleep habits gained 2.4 years compared with those who did none or only one.

 

i) Put this into a moderator sentence: _____ moderates the relationship between _____ and _____ such that ____.

 

Thanks Stephen Chew for sharing yet another great example from the "correlation is not causation" files.

 

Selected answers

a) The headline is causal because it takes the form of advice: If you do X, it will cause Y (here, If you sleep better, it will cause increased life span)

b) Type of sleep and lifespan are the two variables.

c) You might suspect this headline is based on a correlational study (rather than an experiment) because lifespan is definitely a measured variable (you can't manipulate it), and sleep seems likely to be measured as well. Of course, it's possible to randomly assign volunteers to one of two sleep patterns---you'll want to read the article closely to see!

d) One variable is irregularity in sleep timing and duration; the other is metabolic abnormalities. 

e) The study can support covariance because the results showed a correlation between sleep irregularity and metabolic abnormalities. The study may not have temporal precedence--it's not clear if the variables were measured at the same time, or if sleep irregularity was actually measured first. What about internal validity? It's possible that some pre-existing medical condition (such as depression or addiction) goes with both sleep irregularities and with metabolic abnormalities.  Other third variables might work, too--make sure you explain how the variable you are thinking of links to both sleep irregularities and with metabolic abnormalities.

f)

Variable name	How was this variable operationalized?	What are the variable's possible levels?	Is this manipulated or measured?
Healthiness of sleep habits	Survey; summed five habits (falling asleep easily, staying asleep, etc.)	0 to 5	Measured
Early death	National Death Index records	Alive or dead	Measured
Alcohol consumption	Doesn't say how this was operationalized	Not clear--lo to hi	Measured
SES	Doesn't say how this was operationalized	Not clear--lo to hi	Measured

 

g) It meets covariance because the results show an association between healthiness of sleep habits and early death. It meets temporal precedence because the sleep habits were measured first, and then people were followed for four years, when early death was measured. 

h) This argument doesn't work as a third variable critique because the researchers controlled for alcohol consumption in their multiple regression (The CNN article said, "The team then factored out other potential causes for a higher risk of dying, such as alcohol consumption, lower socioeconomic status and existing medical conditions").

 

i)  Gender moderates the relationship between sleep quality and early death such that the relationship is present for both, but is stronger for men than for women. 

 

Personality correlates of liking spicy foods

Photo credit: marcoscastillo/Deposit photos

Here's another article from the content farm website, StudyFinds.  This one's on the personality correlates of liking spicy foods, and it includes a combination of frequency and association claims. The same poll was also covered by the NYPost and by Yahoo! News (with a video)

Below I will post a series of quotes and questions from the StudyFinds version of the story. 

Ninety-three percent of respondents prefer some level of heat in their food. However, nearly four in 10 (36%) say they go for medium heat, 33 percent go for mild, and just 24 percent like their spice level set to hot.

a) Based on this statement, there seem to be four levels of the "Liking Spicy Foods" variable. What are they? 

The methodology of the survey is not always clear from the press stories. Here is one example:

Spicy food eaters are more likely to enjoy trying new things (76%), consider themselves attractive (62%), and are more content with their lives (66%) than those with milder heat preferences.

b) In the quote above, what is your guess about who counts as a "spicy food eater"? That is, which answer would a person have to give to the "Liking Spicy Foods" variable you listed in a) in order to be counted as a "spicy food eater"?

In the quote above, another problem is they only report the percentages for the "spicy food eaters". We'd also need to know the corresponding percentages for "those with milder heat preferences."

c) What would these corresponding percentages be able to tell us about effect size? Propose some alternative outcomes for the variable "trying new things" that would show a large effect size vs. a small one. 

Here is a quote from later in the story:

Almost one-third (32%) of spicy food fanatics follow a certain diet like vegan or vegetarian— more than double than no spice fans (13%) and more than any other spice preference.

It's great that this quote at least tells us the percentage in each group that are vegan/vegetarian. However, it still does not tell us how they operationalized being a "spicy food fanatic" vs a "no spice fan."  

d) Here's another quote from the journalist's story. Based on what you answered so far (in previous questions), what is good and bad about the coverage in this quote? 

Of those who prefer hot foods, 21 percent consider themselves extroverted, compared to 15 percent of mild food eaters. Those who pack on the heat are also more likely to describe themselves as creative (54%), confident (51%), and adventurous (44%).

Another consideration here is the cultural background of the participants. One of my colleagues grew up in India, where children often learn to eat spicy foods from an early age.

e) How might culture moderate the relationships reported here, for example, how might it moderate the relationship between liking spicy food and being extroverted? 

f) This study was conducted by a commercial polling firm, and this particular study was commissioned by the brand Frank's RedHot. Do you think this commercial funding should affect our interpretation of the data? Why or why not? 

Selected Answers:

a) The variable "Liking Spicy Foods" has four levels: No heat, mild heat preference, medium heat preference, and hot heat preference.

b) It's not clear--but it's possible that they counted anybody who picked mild, medium, or hot (that is, 93% of the sample) as a "spicy food eater"?

c) Here's an example. What if 76% of spicy food eaters like trying new things but 70% of non-spicy food eaters like trying new things? That's not a very big effect size! But if 76% of spicy food eaters like trying new things but 20% of non-spicy food eaters like trying new things, that would be a very large effect size. 

d) This quote has the same problem of not defining what a spicy food eater is. But at least they give the percentages of extroverts for two different groups.

e) If a culture serves a lot of spicy foods and even children learn to enjoy them, then it's possible that "liking spicy food" will not be strongly correlated with personality (there's a ceiling effect on the spicy food variable). The personality correlations may exist only for cultures where spicy foods are more variable.

 

 

 

 

 

 

 

What your taste in music says about your personality

According to the study, which personality trait is associated with upbeat, danceable music? Credit: Prostock-studio/Shutterstock

This splashy heading from Inc.com reads:

Here's what your taste in music says about your personality, according to a huge new study. whether you like Mozart or metal really does reveal something about your personality, new research finds

 

This was a correlational study, and provides a good chance to practice concepts from Chapter 3 and 8. Here is how the journalist summarized the methods and results:

...researchers from University of Cambridge and Israel's Bar-Ilan University used an online quiz to extract information on the musical preferences and personalities of more than 285,000 study subjects from 53 countries (you can try the quiz yourself here). They also set up a website where 71,000 visitors from 34 countries rated music clips and took a personality quiz. 

The team used well-known, scientifically validated frameworks for categorizing both music and personality. The aptly named "Music" framework categorizes music using five dimensions: mellow, unpretentious, sophisticated, intense, and contemporary. The familiar Big 5 personality framework categorizes people according to openness, conscientiousness, extroversion, neuroticism, and agreeableness. 

Did the research team find any correlations between the type of music someone enjoyed and their personality type? Yup, reported the researchers recently in the Journal of Personality and Social Psychology. 

a) You could think of this study as having two big variables or ten smaller variables. What would be the two "big" variables? What would be the ten smaller variables? 

b) What makes this a correlational study? 

c) Here is a description of one of their results:

Extroverts the world over tend to like upbeat, danceable contemporary music. 

Sketch a well-labelled scatterplot of this correlation. 

d) Here is another result: 

The study found fans of the Nirvana classic "Smells Like Teen Spirit" and other intense music are higher in neuroticism on average.

Sketch a well-labelled scatterplot of this correlation. 

e) Here is a third result:

Those high in conscientiousness tended to avoid more intense music...

Sketch a well-labelled scatterplot of this correlation. 

f) And here's one more:

Looking for an agreeable type? Then put on some Marvin Gaye and see how they react. The researchers found a strong correlation between liking mellow music and being high in agreeableness.

 

g) Now, here are two quotes from the journalist's article. For each quote decide which of the four big validities this point is addressing.

 

1. The team used well-known, scientifically validated frameworks for categorizing both music and personality. 

2. They ... set up a website where 71,000 visitors from 34 countries rated music clips and took a personality quiz. 

 

h) What is a question you could ask about this study's statistical validity?

 

i) Inc.com is a business magazine, so they framed the study in terms of job hiring:

 

Despite its sometimes dubious scientific basis, the personality testing industry rakes in around $2 billion annually telling companies their new hire is an INFJ or revealing her Hexaco type. [This] study suggests your business might be able to save some money and just ask candidates about their taste in music instead. 

What do you think? Does the research study support the idea that we could use music preferences in job interviews to select certain personality traits? Why or why not?  (to read the Inc.com take on this question, read the full article).