Chapter 08; Bivariate Correlation Research Feed

Should scientists change out of pajamas?

Week 230_shutterstock_1689643582
Does the study's design support the causal claim the journalist attached to it?
Photo credit: R.O.M/Shutterstock

Some journalists simply cannot resist attaching a causal claim to a study where it doesn't belong. Here's an example from the trade magazine, Inside Higher Ed, which reprinted a piece from Times Higher Education. The headline reads, "Scientists urged to change out of pajamas." 

The first paragraph begins:

Academics who are tempted to remain in pajamas during the working day should think again, says an Australian study that has linked the practice to a deterioration in mental health.
 
a) There are actually two claim-like statements in the first paragraph. What are they? (Remember that advice is a form of causal claim). 

b) What seem to be the two variables in the association and causal claims above? 

The journalists summarizes the study this way: 

The study, which drew on 163 responses from staff at five medical institutes in Sydney, also delved into the reality of working at home for researchers. [...]

Some 28 percent of scientists said they wore pajamas at least once a week -- a cohort who were twice as likely to report worsened levels of mental health than those who dressed normally each day, according the study, by David Chapman and Cindy Thamrin [...]

Given the survey format of this study, we can assume that all the variables in this study were self-report. To learn more about this relatively simple study, you can visit the open-access original journal article here, and you can see the full, original text of the survey here

Let's work through the four big validities. 

c) Construct validity: Look through the original survey (here) and find the variables that measured pajama-wearing and mental health assessment. What do you think--how well do these variables seem to be measured? (Guess what? You're assessing face validity.) 

d) Construct validity: When you view the original text of the survey (still here)  you might be a bit surprised by the lighthearted tone of some the survey questions. For example, they ask about the "typical home working environment" they included the option, "hiding in the bathroom." When they asked people what they wear during remote meetings, one of the options was, "none of your business, camera turned off."  How might this casual tone affect the construct validity of the variables being measured? Do you think it will it affect the accuracy of self-reports?

e) External validity: The journalist refers to "scientists", so that seems to be the population of interest. The sample was described this way by the journalist:

The study, which drew on 163 responses from staff at five medical institutes in Sydney...

In the empirical article, there is more detail about the sample: 

An invitation to participate was emailed to all staff, students and affiliates of the Woolcock Institute of Medical Research, Sydney, and later extended to other medical research institutes in Sydney (Garvan Institute, Children’s Medical Research Institute, Centenary Institute, Brain and Mind Centre).

Can this study generalize from this sample to the population of interest? Why or why not? (Don't be tempted to focus on the size of the sample here. Remember that it's not the size that matters for external validity)

f) Statistical validity: Here is the effect size of the relationship between wearing pajamas and mental health: 

[people who wore pajamas at least once per week] were twice as likely to report worsened levels of mental health than those who dressed normally each day

In the empirical article, the rates of poorer mental health were described as 59% (for pajama-wearers) vs. 26% (for non-wearers). What do you think of the strength of this effect? What more information would you like that is relevant to statistical validity?

g)  Internal validity. Let's evaluate whether the study can support the causal claim, "wearing pajamas put the scientists at risk for poorer mental health?" In order to support a causal claim, we need to conduct an experiment. Was this an experiment or a correlational study?  Explain your answer. 

h) Internal validity: Now let's apply the three criteria for causation. 

The study does show covariance, because people who report working in pajamas did have twice the rate of mental health decline, compared to those who did not.

What about temporal precedence? Does the method establish which variable came first in time? 

What about internal validity? Can you think of a third variable (some "C" variable) that might be associated with both wearing pajamas and having worse mental health? 

Now make a conclusion--does the study support the advice to "change out of pajamas?"

i) How might you design a true experiment to study the potential causal effect of pajamas on mental health? What variable would you need to manipulate? What variable would you need to measure? 

 

 

 


03/20/2021

01/20/2021

11/10/2020

10/20/2020

08/20/2020

08/10/2020

07/10/2020

05/20/2020

01/10/2020

12/10/2019

This blog accompanies
Research Methods in Psychology:
Evaluating a World of Information
by Beth Morling

Research Methods in Psychology_9780393536294

View Sample Chapter

Instructors: Request an Exam Copy

Package
SPSS for Research Methods:
A Basic Guide
for FREE with
Research Methods, 4e!

Research Methods in Psychology_9780393935462

Want to Contribute?

If you’re a research methods instructor or student and would like us to consider your guest post for everydayresearchmethods.com, please contact Dr. Morling. If, as an instructor, you write your own critical thinking questions to accompany the entry, we will credit you as a guest blogger.