Everyday Research Methods

Which comes first? Cross-lag study on depression and social media use

Which comes first, social media use or depression? Photo:Africa Studio/Shutterstock

We have seen multiple studies that have tried to estimate how strong the link is between social media use and depression among young adults. We've covered some of these studies on this blog, including the effect size of the correlation and even a brain imaging study that manipulated "likes."

One challenge with studying real-world links between social media use and depression is that most use correlational designs. Therefore, we can't be sure whether any link between social media use and depression means that social media leads to more depression, or if depression leads to more social media use. And of course, in correlational studies there may even be a third variable, such as age, which is associated with both variables and causing them to appear correlated. 

The latest study used a longitudinal, cross-lag panel design, such as those described in Chapter 9, to study this question. Here's a writeup of the study by a journalist at Earth.com.

First, here are some comments about the design: 

The researchers point out that previous studies that found links between depression and social media use among tweens are limited in their results because they only captured a snapshot of teen social media use.

“You have to follow the same people over time in order to draw the conclusion that social media use predicts greater depressive symptoms,” said Taylor Heffer, the lead author of the study. “By using two large longitudinal samples, we were able to empirically test that assumption.”

Here are some more details about the method. They actually used two longitudinal samples: 

For this study, the researchers conducted a two-year longitudinal study surveying 6th, 7th, and 8th graders in Canada. The team also surveyed a series of undergraduate students once a year for six years as they progressed through college.

What were the results? 

Higher depression symptoms predicted social media use among teen girls which indicates that unhealthy social media use may be a symptom depression, not a catalyst for it.

“This finding contrasts with the idea that people who use a lot of social media become more depressed over time. Instead, adolescent girls who are feeling down may turn to social media to try and make themselves feel better,” said Heffer.

Questions

a) In a cross-lag panel design, researchers measure the same two variables in the same sample of people over time. What are the two variables they focused on here?

b) Sketch a series of boxes depicting the design of the study, similar to the figures in Chapter 9's cross-lag panel section. How many time periods would you need to depict for the undergraduate sample? 

c) Focus only on the girls. What do you think the autocorrelations might have been in this design? Fill those in on your sketch.

d) Still focusing only on the girls, given the description of the results, estimate what the two cross-lag correlations might have been. Which correlation should be larger--the one from Time 1 depression to Time 2 social media, or the one from Time 1 social media to Time 2 depression?

 

How much do chimpanzees cooperate?

Which behaviors were used to operationalize cooperation in chimps?
Photo credit: apple2499/Shutterstock

Human beings are a fundamentally prosocial and cooperative species, but what about one of our close relatives, the chimpanzee?  A science writer summarized a series of studies on chimpanzees who live in the Tchimpounga Chimpanzee Sanctuary, in the Republic of the Congo. The summary is here and you can read the empirical journal article here.

The studies illustrate how researchers can operationalize a variable like "cooperativeness" in other species. It provides a clear example of observational research, too. 

Researcher Alexandra Rosati conducted three observational tasks on the sanctuary, all meant to operationalize the construct cooperativeness. The first task took place with chimp volunteers who were alone in a dormitory area. One experimenter grabbed another one's stick and threw it out of reach into the dormitory. The second researcher made a show of trying to get the stick back through the bars of the dormitory. Would the chimp help her out by bringing her the stick? The researcher conducted 10 trials with each chimp. You can see the task in action at this link--watch Video 2. The journalist writes:

On average, the chimpanzees, who received no reward for cooperating, chose to hand the out-of-reach object to the researcher just over 50% of the time – but the rate of cooperation varied significantly among the individuals, with some helping in all 10 trials and others not at all. Chimps who chose to help more often also chose to help more quickly, supporting the conclusion that more cooperative individuals tend to make decisions faster than more selfish individuals do, the authors wrote.

Notice that there are two findings here. The first is the simple result of how many chimps helped (that's a frequency claim result).The second result is a correlational pattern. 

a.) What is the frequency claim you can make here? 

b.) Reread the description of the "stick" study (above) and decide what two variables are correlated.

c.) Placing these two variables on the x and y axis, sketch a scatterplot of the result that is described. (Compare your sketch to Figure 2a in the empirical journal article.) 

d.) Comment on the operationalization of the construct "Cooperation". Why might the researchers have chosen this operationalization? 

Here's a second way they studied chimp cooperation. This operationalization asks whether the chimps would punish individuals who don't follow rules:

When they had an opportunity to punish a thief who stole food from them, the chimps often chose to collapse the experimenter’s table [by pulling out the table's false leg] to prevent the thief from consuming the ill-gotten treats. 

e.) You can watch the "stealing" task here, under Video 3. What behavior (by the chimps) was used to operationalize "cooperativeness" here?  Why might this behavior be considered as an operationalization of Cooperation?

Finally, another task used in the study presented chimpanzees with two options for food. This task is viewable in Video 1 here: 

When presented with two sets of plates, one that contained a snack for both the chimpanzee and a familiar caretaker and another that had a treat only for the chimp, the apes were basically indifferent to whether the experimenter got something to eat. 

f.) The three operationalizations of cooperation in chimps found variable results. Two operationalizations showed some "cooperation," and one apparently did not. What does this pattern of results suggest to you about the importance of how well we operationalize variables? 

 

 

Reading vs. listening

Photo: Ladida/Getty Images

Audiobooks are more popular than ever. Have you ever wondered if there are cognitive differences between reading print books and listening to audiobooks? In this editorial, psychologist Dan Willingham explains some of the research comparing the two formats. Dr. Willingham describes several research findings, most of which come from experimental designs. 

Here's one point Willingham makes: 

...there are differences between print and audio, notably prosody. That’s the pitch, tempo and stress of spoken words. “What a great party” can be a sincere compliment or sarcastic put-down, but they look identical on the page. Although writing lacks symbols for prosody, experienced readers infer it as they go. In one experiment, subjects listened to a recording of someone’s voice who either spoke quickly or slowly. Next, everyone silently read the same text, purportedly written by the person whose voice they had just heard. Those hearing the quick talker read the text faster than those hearing the slow talker.

a. In the study above, what is the independent variable?  Was it manipulated independent groups or within groups (and what keywords in the description help you decide?)

b. What is the dependent variable? 

c. Sketch a small graph of the results described, labeling your axes mindfully.

 

Here is another study from Willingham's summary:

It sounds as if comprehension should be easier when listening than reading, but that’s not always true. For example, one study compared how well students learned about a scientific subject from a 22-minute podcast versus a printed article. Although students spent equivalent time with each format, on a written quiz two days later the readers scored 81 percent and the listeners 59 percent.

d. In this study, what is the independent variable?  Was it manipulated independent groups or within groups (and what aspects if the description helped you decide?)

e. What is the dependent variable? 

f. Sketch a small graph of the results described, labeling your axes mindfully.

g. Do you think you could have conducted this study as a repeated-measures design? Why or why not? 

h. What questions do you have about the internal validity of this study? 

 

Suggested answers

a. The IV is reading speed of the narrator during the first task; its levels are Slow and Fast. The phrase, "either spoke quickly or slowly" tells you this was manipulated as independent groups.

b. The dependent variable was how fast people read the text passage. 

d. Here the IV is reading a text or listening to a podcast. The results phrase refers to "readers" and "listeners," implying that the IV was manipulated as between groups (independent groups).

e. The DV is score on a comprehension quiz.

g. It might be difficult to conduct this exact study as a repeated-measures design, because people would have to process the same content aurally and via reading. Once they've processed the content once, it would undoubtedly be easier to process it a second time. 

h. You'd want to be sure that i) people were randomly assigned to the two conditions. This would control for selection threats; and ii) people in the two conditions read or listened to the exact same text (for example, a podcast and its transcript). This would help avoid a design confound.  

Can puppy pictures help your relationship?

Does pairing this cute puppy with a photo of your partner help 
improve your relationship?
Photo credit: Matanya/Wikimedia Commons

Professor James McNulty, a social psychologist who studies relationships, was asked by the Department of Defense to develop an intervention to help couples cope with the separation of a deployment. How can couples stay close when their partner is far away for months at a time? 

McNulty and his team of students tested an intervention in which couples viewed photos of their partner placed next to cute images of puppies, kittens, or baby bunnies.  Would just seeing your spouse next to cuteness make you feel closer to your spouse? Here's how a journalist from National Geographic described the study: 

For six weeks, one group of partners was shown a stream of images every three days that repeatedly paired their partner's image with that of a positive image, like a puppy. The control group saw pictures of their partners paired with neutral images, such as a shirt button.

Every two weeks [beginning at the start of the study], the couples' attitude toward each other were measured by the positive and negative words used when reviewing a series of faces, of which their partner was included. When couples were evaluated at the end of the study, those who had been shown positive images had more positive associations with their partners and improved marriage quality.

There were 144 couples in the study. You can read the empirical journal article here, in the journal Psychological Science. 

Questions.

a) McNulty and his team conducted an experiment. What are the independent and dependent variables? Was the independent variable manipulated as within groups or independent groups? 

b) Which type of experiment does it appear to be, based on the description: Posttest-only? Pretest-posttest? Concurrent measures? Repeated measures? 

c) Think about the design of this study and consider the 12 internal validity threats presented in Table 11.1. Which threats can you probably rule out, given the design of this study? Which threats do you think might still be in question? What questions might you ask about the study to assess these remaining threats to internal validity? 

 

Suggested answers:

a) The independent variable was which type of treatment they were in: Pairing their partner's face with cute animals or with neutral objects. This was an independent groups IV. The dependent variables included the positive and negative emotions people felt, and their marriage quality. That is, there were multiple dependent variables in the study.

b) This appears to be an independent groups design; specifically, a prettest-posttest design.

c) The researchers randomly assigned couples to the two groups, so we can rule out selection effects. It's not a repeated measures design, so we don't need to worry about order effects. Because they used a control group and an experimental group, we can rule out threats such as maturation, testing, regression, instrumentation, and history, because these threats would have applied equally to both experimental and control groups and thus evened out. We might ask about attrition (selection-attrition) in the two groups in case the unhappiest couples dropped out of the experimental group more often (however, if attrition was similar in the two groups then attrition is not an internal validity threat). We might wonder if some couples guessed the hypothesis of the study and changed their answers (experimental demand). Since both groups received a treatment (either the cute photos or the neutral photos), we can probably rule out a placebo effect. The dependent measures appear to be self-report, so observer bias is unlikely here. 

How well do fire alarms work for kids?

 Which fire alarm worked best to wake young children up? Photo: YMHhappyfamily/Shutterstock

An important applied psychology question concerns smoke alarms. Does the loud, screeching sound of a traditional smoke alarm work on kids? If not, what would work better? There's a Reuters press release about the findings of the study. 

Sleeping children, as it turns out, are fairly impervious to the screeching of a smoke alarm. Researchers found that most children ages 5 through 8 took more than five minutes to wake up with a standard alarm, as compared with around four seconds when they heard the sound of their mother’s voice, according to the results published in The Journal of Pediatrics.

“The thing that was most remarkable to us was to see a child sleep five minutes through a very loud high-pitched tone, but then sit bolt upright in bed when their mothers voice sounded through the alarm,” said the study’s lead author, Dr. Gary Smith.

In this research, experimenters invited families into a sleep laboratory on four different nights. Here's how a journalist summarized the study: 

Smith and his colleagues recruited 176 children between the ages of 5 and 12 and brought them into the sleep lab for testing. The lab had rooms that were set up to look like a typical bedroom, Smith said.

The children’s brain activity was monitored with electrodes on their scalp and face, and they were allowed to fall into the deepest stage of sleep. Then one of four types of simulated smoke alarm was sounded and researchers measured how long it took for a child to wake up and follow a previously rehearsed escape plan.

The experiment was run four times with each child, each about a week apart, so that all the children were exposed to all four types of alarm: standard high-pitched tone, the mother’s voice with instructions like “wake up” and “get out of bed,” the mother’s voice saying the child’s name, and the mother’s voice saying the child’s name and then giving instructions.

There was no significant difference in time to waking between the three alarms that used the mother’s voice. But there was a very large difference between all three of the maternal voice alarms and the standard alarm.

 

Questions: 

a) This is a simple experiment. What is the independent variable? What are its levels? Is it manipulated as independent groups or within groups? 

b) What is the dependent variable? 

c) Which of the four simple experiments is this: Posttest only? Prettest-posttest? Repeated measures? Concurrent measures? 

d) Sketch a graph of the results of this study.

e) What are the advantages to using a within-groups design in this study? 

f) Which internal validity threat do we most worry about in a design like this? How should the researchers control for that internal validity threat?

 

 

Suggested answers:

a) The independent variable is which smoke alarm they tried, and it has four levels: Traditional beep, Mom's voice saying the child's name; Mom's voice saying the name and giving instructions, and Mom's voice giving instructions. It was manipulated within groups. 

b) The DV is time to wake up (and whether they followed the right instructions)

c) This is a repeated measures design. 

d) Sketch should have "time to wake up" on the y axis and the four alarm conditions on the x-axis. 

e) The strongest advantage here is that each child can serve as their own control. Each child's normal level of "wakeability" is held constant across the four alarm conditions. 

f) We mainly worry about order effects in most repeated measures designs. The researchers should counterbalance the order of the four conditions to help control for this threat. To find out if they did that, refer to Method section of the empirical journal article, here. 

"Join a team and get sweaty" to help with depression

Photo credit: isitsharp/Getty Images

A health story's headline reads, "A New Prescription For Depression: Join A Team And Get Sweaty"

Psychologists have known for a while that depression levels are linked to exercise. In fact, some comprehensive treatments for depression include an exercise component in addition to psychotherapy or medication. 

This recent NPR story focused on which type of exercise that might work best: 

..a recent study in the journal Lancet Psychiatry found that popular team sports may have a slight edge over the other forms of physical activity.

The researchers analyzed Centers for Disease Control and Prevention survey data from 1.2 million adults and found — across age, gender, education status and income — people who exercised reported fewer days of bad mental health than those who didn't. And those who played team sports reported the fewest.

a. What are the two central variables in this research, based on what you've read so far?

b. Is this study correlational or experimental? 

c. Sketch a graph of the result. 

d) The journalist writes: 

Now, this study only shows an association between group exercise and improved mental health, and can't prove that the one causes the other. 

Why can't the study support that "one causes the other"?  Apply the three causal criteria (covariance, temporal precedence, and internal validity) to this study. 

 

Anxiously waiting? Tetris might help

 

Which version of Tetris helped the most? Photo: incamerastock / Alamy Stock Photo

NPR's journalists covered a study that tested whether different forms of Tetris would help people feel less anxious.  

The journalist writes that

...research suggests that Tetris can ease us through periods of anxiety by getting us to a blissfully engrossed mental state that psychologists call "flow."

"The state of flow is one where you're completely absorbed or engaged in some kind of activity," Sweeny explains. "You lose your self-awareness, and time is just flying by."

Here's more on the detail: 

Sweeny and her collaborators gathered a group of more than 300 college students and told them their peers would be evaluating how attractive they were. "I know, it's kind of cruel, but we found it's a really effective way to get people stressed out," Sweeny says. While the participants awaited their attractiveness scores, the researchers had them play Tetris.

Some played a painfully slow, easy version of the game — which bored them. Some played an extremely challenging, fast version — which frustrated them. And everyone else played the classic version, which adapts to each player's individual skill level and gets them into that state of flow.[People were randomly assigned to the three groups.]

In the end, everyone experienced a degree of worry. But the third group reported slightly higher levels of positive emotions (on average, about a quarter of a point higher on a five-point scale) and slightly lower levels of negative emotions (half a point lower on a five-point scale).

"It wasn't a huge difference, but we think it's noticeable," Sweeny says. "And over time, it can add up."

Questions:

a) In this study, they decided to manipulate the conceptual variable, "degree of flow." How did they operationalize this variable? 

b) What were the dependent variables in this study? (there seem to be two DVs here)

c) What was the independent variable? What were its levels? 

d) Does this seem to be an experiment or a correlational study? How do you know? 

e) Sketch a graph of the results.

f) The journalist mentions details about the results (e.g., "about a quarter of a point higher on a five-point scale" and "half a point lower on a five-point scale"). Which aspect of statistical validity is being discussed here?

g) What questions would you ask to decide if this study was internally valid? Which of the internal validity threats in Table 11.1 could you rule out? Which could you ask about? 

h) What about the external validity of this study? How might you see if this effect might generalize to other flow-related activities (other than Tetris)? 

 

Corrections on effect size content

Here's an update to some effect size content in the textbook.

Figure S1.12

First, in Supplemental Chapter S1, there's a comparison of psychological to medical effect sizes (page 477 of the 3rd Edition). Figure S1.12 reproduces a chart by Bushman and Anderson (2001) comparing the correlation between violent media and aggression to effect sizes from various medical studies, such as the association between smoking and lung cancer. The chart makes psychology look good--our effect sizes appear larger than common medical wisdom! 

I've learned that this figure is problematic.  One issue with the figure concerns its medical study comparators. The examples suggest that effect sizes for medical treatments and risks are rather small (such as the link between asbestos and cancer or lead exposure and IQ). But over the years, critics have argued that the examples in Bushman and Anderson's figure are artificially low or inaccurately computed. (Notably, Bushman and Anderson have defended their figure, more than once.)

Another issue is that the effect sizes for violent media and aggression are based on meta-analytic estimates  that vary depending on the techniques used. Certain researchers report medium sized effects, but others argue that media effects on violence are negligible. (This debate led me to remove a media violence meta-analysis from Chapter 14 when I prepared the 3e.)

Ultimately, I am persuaded by Ferguson's (2009) explanation of how effect sizes of medical treatment studies vary dramatically, depending on whether they include "hypothesis irrelevant" cases. Especially when effect sizes are based on population-based epidemiological frequency counts, they can drastically underestimate the effect size of a vaccine, drug, or risk factor. 

Chapter 8: Aspirin and heart attack

Chapter 8 uses the classic example of "aspirin prevents heart attack" as an illustration of how a very small effect size (r = .03) can, in fact, have large life-or-death impact. But if the aspirin-heart attack effect size came from a study that included hypothesis-irrelevant cases, it probably underestimated the true effect size. Indeed, the Physician's Health Study estimated that aspirin reduces the risk of a first heart attack by 44%--a much larger effect. 

As always, I am grateful to colleagues who read my text carefully and alert me to problematic examples.  Thanks!

Friends and brain health

What do you think about the external validity of the study? Photo credit: Rawpixel.com/Shutterstock

This story's headline reads, "social ties could preserve memory, slow brain aging" and it's illustrated with an image similar to the one shown here. The press release sets out the goal of the study:

“We know that in humans there’s a strong correlation between cognitive health and social connections, but we don’t know if it’s having a group of friends that’s protecting people or if it’s that people with declining brain health withdraw from their human connections,” [Study researcher] Kirby said.

[The n]ew research ...found that mice housed in groups had better memories and healthier brains than animals that lived in pairs.

 a) Before reading on, reflect: Why would a researcher probably need an animal model to test this question experimentally? 

 

Here's some more detail about the experiment:

Some mice lived in pairs, which Kirby refers to as the “old-couple model.” Others were housed for three months with six other roommates, a scenario that allows for “pretty complex interactions.”

The mice were 15 months to 18 months old during the experiment – a time of significant natural memory decline in the rodent lifespan.

In tests of memory, the group-housed mice fared better. 

One test challenged the mice to recognize that a toy, such as a plastic car, had moved to a new location. ...“With the pair-housed mice, they had no idea that the object had moved. The group-housed mice were much better at remembering what they’d seen before and went to the toy in a new location, ignoring another toy that had not moved,” Kirby said.

In another common maze-based memory test, mice are placed on a well-lit round table with holes, some of which lead to escape hatches. Their natural tendency is to look for the dark, unexposed and “safe” escape routes.

The “couples” mice didn’t get faster at the test when it was repeated over the course of a day.“But over the course of many days, they developed a serial-searching strategy where they checked every hole as quickly as possible. It’d be like walking as quickly as possible through each row of a parking lot to look for your car rather than trying to remember where your car actually is and walk to that spot,” Kirby said.

The group-housed mice improved with each trial, though. “They seemed to try to memorize where the escape hatches are and walk to them directly, which is the behavior we see in healthy young mice,” Kirby said. “And that tells us that they’re using the hippocampus, an area of the brain that is really important for good memory function.”

b) What was the independent variable in this study? How was it operationalized?

c) What was the dependent variable? What were the two ways it was operationalized?

d) How does this experiment help us decide which comes first--social life or better memory? (note: This is temporal precedence!)

e) Do you think the journalist is justified in generalizing this study's results from mice to older adult humans? Why or why not? 

f) Chapter 3 explains how internal validity and external validity are often in a trade-off. Describe how this study with mice illustrates that trade-off. 

 

A test of dog empathy

Which of the two dependent variables showed that dogs were affected by their owners' sadness? Photo: adogslifephoto/istock/Getty Images Plus

Here's a short video with footage from an experiment that tested dogs' empathy. Watch the video with these questions in mind.

a) The study was an experiment. What was the independent variable (IV)? 

b) Was the IV manipulated as between-groups (independent groups) or within-groups? What keywords in the video description helped you answer this question? 

c) The dependent variable (DV) was operationalized in two ways. What were they? 

d) One of the DVs did not support the hypothesis, but the other DV did. Explain the results that they found. (You can sketch two little bar graphs, too.)

e) What do you think--does "opening a door to release a crying owner" indicate "empathy?" (P.S., That's a construct validity question.)

f) Does the study support the claim that "hearing their owners ask for help while crying causes dogs to help their owners faster"?  Apply the three causal criteria to support your answer. 

 

Suggested answers:

a) The IV was whether the owners were saying "help" while pretending to cry, or saying "help" in a neutral tone, while humming Twinkle Twinkle Little Star.

b) This was independent-groups--the reporter used keywords such as "some owners" and "other owners". This was a posttest-only design. 

c) One operationalization of the DV was whether the dogs opened the door, or not. The other operationalization of the DV was how long each dog took to open the door.

d) Only the "time taken" DV showed the predicted effect.

e) Answers will vary

f)  The results show covariance because dogs whose owners were crying opened the door three times as fast as dogs whose owners were calm. Temporal precedence is ensured by the methodology--by randomly assigning owners to cry (vs. sing), they ensured that this condition came before opening the door. The study would have good internal validity if they randomly assigned owner/dog pairs to the two conditions--this would take care of selection threats such as having more "already helpful" dogs in the crying condition, or having owners who are better at acting in one condition or the other. As far as design confounds, we might ask about whether the owners in the two conditions acted exactly the same in all ways except their assigned conditions.