Everyday Research Methods

Stressed? Writing a to-do list might help

To do: Decide whether each study's variables are manipulated or measured. Check! GoodStudio/Shutterstock

I wouldn't be able to get through my week without a to-do list. And now my habits are validated by this new story about research on to-do lists, which comes from the BBC News. 

The journalist summarizes a series of studies for the article. It provides a great opportunity to practice deciding if a study is correlational or experimental.

Here's one quote: 

In a study of employees at a German IT company, those who had unfinished tasks left over at the end of the working week were more likely to think about their work problems over the weekend compared with those who were more or less on top of their workload. ...Those with unfinished tasks were also more likely to find their sleep was disturbed over the weekend.

a) What are the variables in the study above? What are the levels of each variable? Are the variables manipulated or measured?

b) Given your answers to a) above, Decide whether this study is correlational or experimental. 

 

Next the journalist asks,

So, what can you do to stop these whirring worries from keeping you awake?

You could count sheep. You could read a book for a while [...]. Or you could turn the light on and make a list of all those things you need to do. [...]

 You might think this is the last thing that would work, but  a study conducted in the US found it was surprisingly effective.

In fact, when Michael Scullin, director of the Sleep Neuroscience and Cognition Laboratory at Baylor University in the US asked one group of volunteers to write a list just before bed of everything they'd achieved that day, and a second group to write a to-do list, all about the tasks they had to do complete tomorrow and in the next few days, it was this second group who subsequently fell asleep more quickly.

c) What are the variables in this new study? What are the levels of each variable? Are the variables manipulated or measured?

d) Given your answers in c) above, Decide whether this study is correlational or experimental. 

 

The journalist also gives an indication of the effect size in Scullin's study:

... how much quicker? Nine minutes, no less.

And the journalist also addresses the study's construct validity--citing its  physiological variable: 

The scientists didn't rely on the volunteers' own assessment of their sleep (which isn't always accurate), but confirmed when they were awake and asleep through a type of sleep study known as polysomnography. This involves attaching sensors to a person's head and to other parts of the body in order to monitor their brainwaves, breathing and movements.

 

For more practice, here's one more study on to-do lists, as described by the journalist:

It is best to list every specific task, rather than to use general headings, even though it will make your list longer. Professor Scullin's study found that busy people who created lists of more than 10 tasks fell asleep an average of 15 minutes faster than people who didn't write out to-do lists. They also fell asleep six minutes faster than those who only compiled short lists. So, make it comprehensive.

e) What are the variables in this study? What are the levels of each variable? (Hint--the IV here has three levels--be sure you name all three!) And, are the variables manipulated or measured?

f) Given your answers in e), Decide whether this study is correlational or experimental. 

g) Sketch a bar graph of this study's results.

h) Can this study support the journalist's claim that "It is best to list every specific task, rather than to use general headings"? Why or why not? 

The most fulfilling jobs in America

The story reported that members of the clergy are most likely to score high on all four questions.
Photo: Diego Cervo/Shutterstock

Before you read on, make a prediction: Which U.S. jobs would you predict would be the most fulfilling?  In this story from the Washington Post column,"Department of Data" you can explore some fascinating data on U.S. job satisfaction.

In a fun twist, the article also talks about how the construct of "fulfilling work" is measured. (It's not everyday that we get to celebrate construct validity: hooray!)

The data come from the CEV--the Civic Engagement and Volunteering Supplement, which is part of the Census Bureau’s Current Population Survey.

Here's how the journalist introduces the data:

In 2021 and in 2023, the researchers behind the CEV asked whether you agree or disagree with these four statements:

• I am proud to be working for my employer.
• My main satisfaction in life comes from work.
• My workplace contributes to the community.
• I contribute to the community through my work.

The two years of surveys give us enough responses to start doing some serious analysis — or, in the grand tradition of this column, some less-than-serious analysis.

The Post column presents data (click here and scroll down) showing that agreement with these four items increases with age and with education. 

In general, the figures and charts of this story are fun to explore--but they're under copyright, so I can't paste them here. Here are some of the journalist's summary points. You'll see that which profession comes out on top depends on the question. 

The workers most likely to say they are proud to be working for their employer and that they gain satisfaction from work are — surprise! — the self-employed.The self-employed who are incorporated — a group that often includes small-business owners — are almost twice as likely as private-sector, for-profit workers to strongly profess pride in their employer.

Government and nonprofit workers fall somewhere in the middle on those [pride and satisfaction] questions. But they rank at the very top on “My workplace contributes to the community” and “I contribute to the community through my work.” Local government workers, who include teachers, take the top spot for strong agreement on both, followed by nonprofit workers. Private-sector, for-profit workers again lag behind.

The jobs that do worse on these measures [of contributing to the community] tend to be in manufacturing or other blue-collar production and extraction jobs, or at the lower-paid end of the service sector. Folks in food services (e.g., bartenders and food prep), janitorial roles and landscaping, and personal services (e.g., barbershops, laundry and hotels) all struggle to find greater meaning in their work. Though some in better-paid service jobs also struggle by some measures — think sales, engineering or software development.

The story indicates that members of the clergy report the most positive answers across all four of the satisfaction items.

a) Do the four items used in this story (included above in this blog post) have face validity as a measure of job satisfaction? Why or why not?

b) Click to the story and scroll to find the line graph that shows agreement with the four items over the lifespan. What do you notice? What do you wonder?

c) One of the concepts in Chapter 5 is internal reliability, or internal consistency, which is relevant when researchers use multiple items to measure the same thing. In this story, the journalist kept the four satisfaction items separate. But based on what you see in the two line graphs, do you think the four items would have internal reliability? That is, would they likely correlate with each other and have a strong Cronbach's alpha?

d) Students often mistrust self-report questions like these. However, in my opinion, the demographic and job category data provide criterion validity evidence that these four questions are construct valid--that is, that people are not just lying or making things up. Specifically, I think the data shows known groups criterion validity for these four items. Can you explain why? Review what known groups criterion validity is, and apply that definition to this new example. 

e) A friendly reminder: Frequency claims (such as "local government workers are most likely to think their workplace contributes to their community" need not just construct validity, but also external validity. How would you ask about that for this data set? Do you think this data set is externally valid--why or why not? 

For even more fun, the column also includes some charts from a different set of data, showing the happiest, most meaningful, and most stressful daily activities (including spiritual pursuits, volunteering, exercising, and working), as well as charts indicating the happiest places (such as places of worship, one's own home, others' homes, and so 0n). Data-loving psychologists (and research methods students) will enjoy exploring these figures.