Everyday Research Methods

Fidget spinners seem to cause worse memory and attention

The fidget spinner IV was manipulated as independent groups in the first study and as within groups in the second. Photo: Marcel Jancovic/Shutterstock

Fidget spinners were extremely popular a couple of years ago. When people promoted them, they often suggested that the toys could help people with ADHD focus and learn more. At the time, such claims were not scrutinized empirically (here's a nice summary of this lack of evidence). It takes time for empirical research to be conducted, analyzed, peer-reviewed, and published, and even though fidget spinners were hot in 2017, we now have a study that tested the effect of using a fidget spinner during a learning task. At least one press outlet covered this study. Here's their summary:

In the first experiment, 98 undergraduate students watched a 15-minute educational video lecture on baking bread while either using a fidget spinner or not using a fidget spinner [randomly assigned]. In addition, half of the participants not using a fidget spinner were randomly assigned to watch the lecture while sitting near someone using a fidget spinner.

A fill-in-the blank memory test showed that participants who used fidget spinners answered significantly fewer questions correctly about the material covered in the video. Participants who used fidget spinners were also more likely to report “zoning out” and having difficulty staying on task while watching the video.

Participants who were sitting near someone using a fidget spinner, however, did not appear to be affected.

Questions

a) This was an experiment. How do you know?

b) What were the IV and DV? How many levels were in the IV and what were they? Was the IV manipulated as independent groups or within groups? 

c) Which of the four simple experiments is this: postttest-only? Pretest-posttest? Repeated-measures? Concurrent measures? 

d) The journalist attached a causal claim to these results, saying "study finds using a fidget spinner can impair memory and attention." Apply the three criteria for causation to this claim to explain why the causal claim is probably valid. How does random assignment assist this study's internal validity?

e) Ask one question each about the study's statistical, construct, and external validities. 

The researchers replicated the effect in a second study. The wondered if fidget spinners might only work among people who believed in their powers: 

The researchers then conducted the experiment again. However, this time around they specifically recruited 48 undergraduates who believed that fidget spinners could help them focus in class.

But the results were the same.

According to the empirical journal article, the second study on 48 students was conducted as a repeated measures design, using two videos. Students used a fidget spinner during one video and no spinner during the other one. Students were tested on their recall for the videos after each one.

f) What are the IV and DV in the second study? How many levels were there in the IV?

g) How do you think the researchers used counterbalancing in this second study?  Why would that have been important? 

You can access the original empirical article here, though you may need to log in via your university library. 

 

 

 

 

 

Articles that are student-friendly

A list of recent articles that are student-accessible 

Compiled by Beth Morling and Jeong-Min Lee, Georgia State

Our students have used these articles in their final projects for Research Methods in the past. In this final project, students summarize and analyze the validities for two empirical studies. (I can provide the assignment upon request: morling@udel.edu.)

Most of these articles report more than one study, and at least one of the studies is simple enough for a student with basic skills in research methods to understand. The most complex experimental design would be a 3-way factorial design. The most complex correlational design uses multiple regression and (sometimes) mediation.

 

Brummelman, E., Thomaes, S., de Castro, B. O., Overbeek, G., & Bushman, B. J. (2014). “That’s not just beautiful—that’s incredibly beautiful!”: The adverse impact of inflated praise on children with low self-esteem. Psychological Science, 25, 728-735. doi:http://dx.doi.org/10.1177/0956797613514251

Bryan, C.J., Yeager, D.S., Hinojosa, C.P., Chabot, A., Bergen, H., Kawamura, M., & Steubing, F. (2016). Harnessing adolescent values to motivate healthier eating. PNAS, 113, 10830-10835. doi: 10.1073/pnas.1604586113

Cantú, S. M., Simpson, J. A., Griskevicius, V., Weisberg, Y. J., Durante, K. M., & Beal, D. J. (2014). Fertile and selectively flirty: Women’s behavior toward men changes across the ovulatory cycle. Psychological Science, 25, 431-438. Retrieved from http://search.proquest.com/docview/1501839110?accountid=10457

Chan, K. Q., Tong, E. M. W., & Tan, Y. L. (2014). Taking a leap of faith: Reminders of God lead to greater risk taking. Social Psychological and Personality Science, 5, 901-909. doi: 10.1177/1948550614537309

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116(39), 19251–19257. https://doi.org/10.1073/pnas.1821936116

 Fraley, R.C., Griffin, B.N., Belsky, J., & Roisman, G.I. (2012). Developmental antecedents of political ideology: A longitudinal investigation from birth to age 18 years. Psychological Science, 23, 1425-1431. doi: 10.1177/0956797612440102

Francis, H. M., Stevenson, R. J., Chambers, J. R., Gupta, D., Newey, B., & Lim, C. K. (2019). A brief diet intervention can reduce symptoms of depression in young adults – A randomised controlled trial. PLOS ONE, 14(10), e0222768. https://doi.org/10.1371/journal.pone.0222768

 Frenda, S., Patihis, L., Loftus, E., Lewis, H., Fenn, K.  (2014) Sleep deprivation and false memories. Psychological Science, 25, 1674-1681. doi: 10.1177/0956797614534694

 Gino, F., & Wiltermuth, S. S. (2014). Evil genius? How dishonesty can lead to greater creativity. Psychological Science, 25, 973-981.doi: 10.1177/0956797614520714

Gino, F., Norton, M. I., & Ariely, D. (2010). The counterfeit self: The deceptive costs of faking it. Psychological Science, 21, 712-720. doi:http://dx.doi.org/10.1177/0956797610366545

Gunraj, D. N., Drumm-Hewitt, A. M., Dashow, E. M., Upadhyay, S. S. N., & Klin, C. M. (2016). Texting insincerely: The role of the period in text messaging. Computers in Human Behavior, 55, 1067–1075. https://doi.org/10.1016/j.chb.2015.11.003

House, J., DeVoe, S. E., & Zhong, C. (2014). Too impatient to smell the roses: Exposure to fast food impedes happiness. Social Psychological and Personality Science, 5, 534-541. doi:http://dx.doi.org/10.1177/1948550613511498 [use study 3]

Jirout, J. J., & Newcombe, N. S. (2015). Building blocks for developing spatial skills: Evidence from a large, representative U.S. sample. Psychological Science, 26, 302-310. doi:http://dx.doi.org/10.1177/0956797614563338

 Kruse, E., Chancellor, J., Ruberton, P. M., & Lyubomirsky, S. (2014). An upward spiral between gratitude and humility. Social Psychological and Personality Science, 5, 805-814. doi:http://dx.doi.org/10.1177/1948550614534700 [Study 2]

Lee, K., Talwar, V., McCarthy, A., Ross, I., Evans, A., & Arruda, C. (2014). Can classic moral stories promote honesty in children? Psychological Science, 25, 1630-1636. doi:10.1177/0956797614536401

Levy, K. N., & Kelly, K. M. (2010). Sex differences in jealousy: A contribution from attachment theory. Psychological Science, 21, 168-173. doi:http://dx.doi.org/10.1177/0956797609357708

Markey, P. M., Ivory, J. D., Slotter, E. B., Oliver, M. B., & Maglalang, O. (2019). He does not look like video games made him do it: Racial stereotypes and school shootings. Psychology of Popular Media Culture. https://doi.org/10.1037/ppm0000255

 Mrazek, M. D., Franklin, M. S., Phillips, D. T., Baird, B., & Schooler, J. W. (2009). Mindfulness training improves working memory capacity and GRE performance while reducing mind wandering. Psychological Science, 24, 776-781. DOI: 10.1177/0956797612459659.

Nelson, S. K., Kushlev, K., English, T., Dunn, E. W., & Lyubomirsky, S. (2013). In defense of parenthood: Children are associated with more joy than misery. Psychological Science, 24, 3-10. doi:http://dx.doi.org/10.1177/0956797612447798

 Nusbaum, E. C., & Silvia, P. J. (2011). Shivers and timbres: Personality and the experience of chills from music. Social Psychological and Personality Science, 2, 199-204. doi:http://dx.doi.org/10.1177/1948550610386810 [Focus on “personality and chills during music” and “do music preferences mediate effects of openness” results]

 Ronay, R., & von Hippel, W. (2010). The presence of an attractive woman elevates testosterone and physical risk taking in young men. Social Psychological and Personality Science, 1, 57-64. doi:http://dx.doi.org/10.1177/1948550609352807 [focus on “skateboarding” and “testosterone mediation” results]

 Saslow, L. R., Muise, A., Impett, E. A., & Dubin, M. (2013). Can you see how happy we are? facebook images and relationship satisfaction. Social Psychological and Personality Science, 4, 411-418. doi:http://dx.doi.org/10.1177/1948550612460059 [Study 2]

 Seder, J. P., & Oishi, S. (2012). Intensity of smiling in facebook photos predicts future life satisfaction. Social Psychological and Personality Science, 3, 407-413. doi:http://dx.doi.org/10.1177/1948550611424968 [focus on Study 1]

Tice, D. M., & Baumeister, R. F. (1997). Longitudinal study of procrastination, performance, stress, and health: The costs and benefits of dawdling. Psychological Science, 8, 454-458. Retrieved from http://search.proquest.com/docview/619221802?accountid=10457

 Valor-Segura, I., Expósito, F., Moya, M., & Kluwer, E. (2014). Don't leave me: The effect of dependency and emotions in relationship conflict. Journal of Applied Social Psychology, 44, 579-587. doi:http://dx.doi.org/10.1111/jasp.12250 [use Study 1]

 Vezzali, L., Stathi, S., Giovannini, D., Capozza, D., & Trifiletti, E. (2014). The greatest magic of harry potter: Reducing prejudice. Journal of Applied Social Psychology, 45, 105-121. doi:http://dx.doi.org/10.1111/jasp.12279  [Study 1]

Witkower, Z., & Tracy, J. L. (2019). A facial-action imposter: How head tilt influences perceptions of dominance from a neutral face. Psychological Science, 30(6), 893–906. https://doi.org/10.1177/0956797619838762

 Wong, E., Ormiston, M., Haselhuhn, M. (2011). A face only an investor could love: CEOs’ facial structures predict their firms’ financial performance. Psychological Science, 22, 1478-1483. doi: 10.1177/0956797611418838.

Cuter kids get higher grades

One of the proposed mediators of the effect was that teachers report better relationships with attractive students.Photo credit: FatCamera/Getty Images

File this one under: Depressing, but we suspected it all along. The Washington Post ran a summary of research that looked at the link between school achievement and physical appearance. 

A new study finds that good looking kids do better in school than their less striking peers.

The research, by Barnard College economist Daniel Hamermesh and colleagues, finds that people whose looks are “one standard deviation above average” attain nearly five more months of schooling than an “otherwise identical average-looking individual.”

The study in question found the effect in two large, national studies, one in the U.S. and one in the U.K. You might be wondering, how did they decide how good looking the kids were? What do you think of the construct validity of this operationalization: 

For each kid in the American study, looks were assessed by a panel of at least 10 undergraduates who watched segments of video interviews with the children collected over the course of the survey. The raters assessed each one’s looks on a scale from 1 (not at all cute/very unattractive) to 5 (very cute/very attractive).

For the U.K. study, attractiveness was assessed by the children’s teachers, who assigned each child to one of four categories: attractive, unattractive, “abnormal feature” or “underfed or scruffy and dirty.”

Here is a description of the results:

Ratings in hand, the researchers then analyzed the relationship between a child’s looks and academic achievement, as measured via various standardized tests administered throughout the two studies. The studies revealed the same general pattern: Better looking children performed better on tests of academic achievement, even when controlling for ethnicity, gender and parents’ education and income.

The language above should cue you to realize that this study evaluated the core relationship (between looks and achievement) but used multiple regression techniques to control for these potential internal validity threats. 

a) What is the criterion variable in this study? What are the predictor variables?

b) Sketch a little regression table with results that would be consistent with what is mentioned above. Put the criterion variable at the top, and list the predictor variables. Which betas would be significant here? 

(Note: You can see the actual results in Table 7 of the scientific article.)

The journalist continues the summary with this statement:

Economists who’ve studied academic performance have found that children who perform better on tests tend to get more schooling: A student who aces all her reading and math tests is much more likely to go on to college than one who does not. Hamermesh and his colleagues estimate that good-looking kids’ heightened test performance relative to their average-looking peers works out to about 0.4 years, or five months, of additional schooling.

c) The above is a statement about effect size. Do you think this effect is a large one? Is it important?  What more do you need to know to evaluate this effect? 

d) Next comes a statement about potential mediators of the relationship. Read the following and sketch out one of the mediation paths they are describing (use Figure 9.11 as a model)

To determine why, Hamermesh and his colleagues tested a number of theories. They found some evidence that teachers report better relationships with the more attractive students, which explains some of the achievement gap. They also found that youngsters rated as unattractive were somewhat more likely to report being bullied by their peers, with detrimental spillover on their academic performance.

Similarly, they found that there were fewer reports of behavioral problems at school among the highly rated children.

 

Are there other potential mediators you can think of?  

 

Guess what! This is my 200th post! Thanks for reading and making this blog possible. 

Dog people live longer

The study found an association between dog ownership and longevity. Which third variables were controlled for? Which third variables might still be responsible for the relationship? Photo: Shutterstock

I can't resist blogging about research on dogs. Here's a story on NPR that summarized several studies that have tested the correlation between longevity and dog ownership. The journalist introduces the topic this way: 

Dogs and humans that interact with one another get a jolt of oxytocin, the so-called "cuddle hormone." And, if you get to look at dogs and hug them every day, you just might live longer than people who don't have to clean animal hair off their clothes, according to a pair of studies out this month.

Here is a summary of one of the studies:

[researchers] looked over a 12-year period at dog owners in Sweden who have cardiovascular disease. They adjusted for the kinds of things we know affect cardiovascular health — age, demographics, socioeconomic status, marital status, number of children at home — and even after adjusting for all of that, they found a benefit of dog ownership [on longevity].

....they followed over 100,000 people and found a 21% reduction in deaths from any cause.

The study above used multiple regression. 

a) What was the dependent (criterion) variable in this study?  What were the predictor variables? 

b) Sketch a little regression table with results that would be consistent with what is mentioned above. Put the criterion variable at the top, and list the predictor variables. Which betas would be significant here? 

c) The researchers controlled for age, demographics, SES, marital status, and children--but regression doesn't control for all third variables. What else could be associated with both dog ownership and longevity? 

d) They mention a "21% reduction in deaths from any cause."  Which of the four big validities is this concerning? 

One of the experts who was interviewed for this story said:

In particular, I found it very convincing and striking that the benefits seems to be larger among individuals who live alone compared to multiperson households. That suggests the companionship of a dog is possibly very important to their heart health.

e) This is a moderator! Identify the variables in this relationship and complete this sentence: ____ moderates the relationship between ____ and _____.

 

In part of the article, the journalist presents quotes from an expert, who seems to equivocate on the causality issue:

"When you look at the big picture and look at all the evidence around dog ownership and cardiovascular health, it's pretty clear the signal is real and likely causal," he says.

In other words, he's convinced getting a dog improves your health — especially for those with heart disease.

The research is not definitive, though, Kazi adds. "They're not randomized trials, the gold standard for what we would do to evaluate a new drug," he says.

f) It is frustrating to read that the expert believes the "signal is real and likely causal" and then admits that "it's not definitive." You can set this expert straight. Why can't this correlational study support causation?  Apply covariance, temporal precedence, and internal validity. When you address internal validity, check out answer c) above!

 

 

Suggested answers

a) The criterion variable is longevity (being alive at the end of the study). The predictor variables are dog ownership, age, demographics, SES, marital status, and children living at home. 

b) The table might look like this:

DV: Longevity (or being alive after 12 years)

IV                            Beta (sig)
Dog ownership         .12*
Age                            -.11
SES                              .13*
Married                        .08*
Children at home        .07*

d) Current health and mobility might be associated with both dog ownership and longevity. After all, you  may need to have a certain level of health and mobility to own a dog, and healthier folks are more likely to be alive at the end of the study. 

e)  Living status (being alone or married) moderates the relationship between dog ownership and longevity.  

f) There is covariance--there is a link between owning a dog and living longer. 

There is temporal precedence-- the study took place over several years and dog ownership was measured first, and death rates later. 

There is not full internal validity. They controlled for age, demographics, SES, marital status, and children living at home, so they can rule those out. But they did not control for everything. One possibility is that current health and mobility might be associated with both--after all, you  may need to be at a certain level of health and mobility to own a dog, and those folks are more likely to be alive at the end of the study. 

 

 

 

Racial stereotypes behind how we explain school shootings

Participants and newspapers were more likely to blame video games when a school shooter was white than when he was black. Photo: Richard Levine / Alamy Stock Photo

What causes mass shootings? Of course, we have no experimental evidence on this question. Correlational evidence (both within the U.S. and evidence that compares the U.S. to other countries) is consistent with the argument that gun availability is a factor. 

However, this post isn't about what really causes school shootings. It's about what people think causes them. And it turns out that the race of a school shooter influences what people believe about the shooter. 

After some school shootings (a subset of mass shootings), people have sometimes looked for answers not in the availability of guns, but in video games. Now, there is actually no evidence linking video games to mass shootings. However, this ABC news story talks about when people are more likely to think that video games are responsible: 

There's little evidence showing a relationship between violent video games and school shootings, but video games are often blamed for overall violence in our communities. A recent study published in the Psychology of Popular Media Culture set out to find if we look for an external explanation for violence by whites more often than we do for African Americans.

"We try harder to make excuses for white perpetrators," said investigator Dr. James Ivory, a professor in the Department of Communications at Virginia Tech, in an interview with ABC News.

The first study gave people a mock news story about a school shooting. When asked what caused the shooting, people were more likely to blame video games if the shooter was white than if the shooter was black.

It turns out that in the original study (linked here;  if it's paywalled you may be able to access via your university library), participants read the mock news story and were randomly assigned to one of two photos of the perpetrator--a mugshot showing either a Black or a White man. 

a)  There were two variables in this study. Name each one. For each, was it manipulated or measured?

b) What kind of study is this, correlational or experimental?

c) Sketch a graph of the results of this study. What will you put on the y-axis? 

d) In the study above, what role does random assignment play? Why did the researchers randomly assign people to see either a Black or White perpetrator?  

 

Here is how ABC news described the second study:

The second study looked at thousands of news articles -- video games were mentioned more often in stories of white shooters [compared to black shooters]. 

e) What are the variables in the study above? Name each one; for each, was it manipulated or measured?

f) What kind of study is this, correlational or experimental?

g) Sketch a graph of the results of this study.

h) In the study above, the researchers could not randomly assign news article authors to write about a black vs. a white perpetrator. How does that change the impact of this study? 

i) Which of the two studies do you think provides the strongest evidence that race influences people's willingness to think video games are a cause of the shooting?

 

Suggested answers

a) One variable is race of the perpetrator (levels are Black or White), this was manipulated. The other variable is "likelihood to blame video games". This was measured.

b) This was an experiment because one variable was manipulated. 

d) Random assignment contributes to the study's internal validity. They can be sure that the same kinds of participants were in each group (reading about Black vs. White perpetrators). Therefore, people who play a lot of video games themselves were equally likely to be in the two groups. You might be interested to know that in the original study, the researchers also pretested the Black and White photos they used to be sure that the faces differed on race but not on perceived attractiveness or dangerousness. This pretesting also contributed to the experiment's internal validity.

e) The first variable was race of the perpetrator in the article (Black or White), which was measured. The second variable was whether or not the article mentioned video games (yes or no).

f) This was a correlational study because all variables were measured. 

h) Because this study used naturally-occurring crimes, the researchers may have an internal validity problem. It is possible that the Black and White perpetrated shootings differed not just on race but also on number of victims, age of victims, geography, or some other variable. This introduces an alternative explanation for the result--perhaps people blamed video games for White perpetrated shootings because they had more victims, had older victims, and so on.

i) You might have selected the first study as the strongest because it was an experiment and allows us to make a causal inference. Or you might have selected the second study because it was conducted on "real world" stimuli such as news stories--therefore it appears to have good ecological validity or external validity. 

Thanks to Marianne Lloyd for this idea!

How many have Internet access? Question wording matters

CSP_rasstock/agefotostock

What percentage of Americans would you guess have access to the Internet?  How would you know?

When you consider whether people have computers at home, computers at work, smart phones, ortablets, you might predict that a large majority of Americans are able to go online. In fact, it's probably around 90%.  But, as with all frequency claims, the answer to the question hinges on two key things: Construct validity (How well do you measure "access the Internet?") and external validity, ("how well does your sample represent the population?").

In this article, Pew Research explains how they get different estimates depending on how they ask. They write, 

While our approach has evolved over time, the current measurement consists of two questions: “Do you use the internet or email, at least occasionally?” and “Do you access the internet on a cellphone, tablet or other mobile handheld device, at least occasionally?” In a January 2018 phone survey, 89% of U.S. adults said “yes” to at least one of these questions.

They compare their own questions to that of the U.S. Census Bureau's American Community Survey (ACS):

The ACS asks, “At this house, apartment, or mobile home – do you or any member of this household have access to the internet?”

Yes, by paying a cell phone company or Internet service provider
Yes, without paying a cell phone company or Internet service provider
No access to the Internet at this house, apartment, or mobile home. 

This question led to a similar estimate:

88% of American adults had either free or paid Internet access at home.

But what about this question, from the Bureau of Labor Statistics CPS data: 

That survey asks, “Does anyone in this household, including you, use the internet at home?” Based on this question alone, 81% of American adults live in households where someone uses the internet at home. The difference between the two figures can be at least partly attributed to the questions’ wording and scope....  And households can have internet access without anyone actually using it – for example, if it is provided for free by their community or included in a bundle with other services.

The CPS separately asks, “Does anyone in this household access the internet using a data plan for a cellphone, smartphone, tablet, mobile hotspot, or other device? This type of internet service is provided by a wireless carrier, and may be part of a package that also includes voice calls from a cellphone or smartphone.” If we include Americans who say yes to this question, the previous 81% figure rises to 83%.

And what about using the Internet somewhere other than home? 

The CPS also asks a battery of questions about whether anyone in the household uses the internet at work; at school; at a coffee shop or other business that offers internet access; while traveling between places; at a library, community center, park or other public place; at someone else’s home; or at some other location. (Each location is asked about separately.) If we once again expand the previous CPS estimate to include people who may not use the internet at home but do so somewhere else, the estimate ticks up to 85%.

Questions

a) What companies or organizations might be most interested in having an accurate answer to the question of how many Americans are online? 

b) Make a table where you have the wording of a question, followed by the estimate of the number of Americans on the Internet. There seem to be at least four different ways of asking. (You can see trends over time by scrolling down to the bottom of the Pew article.)

c) How important, in your view, are the different ways of asking the question? That is, some might say that all the estimates are virtually the same. But others might say that differences of 1 or 2% percentages matters. What do you think--are the differences in these estimates important or not? 

d) What subgroups of Americans would you estimate are least likely to access the Internet? 

 

 

Can Probiotics Actually Treat Obesity?

 

Should you be suspicious of a study conducted by a company that produces probiotics products? Photo: Science History Images / Alamy Stock Photo

Guest post by Dr. Dina Gohar, University of Michigan

 

Hippocrates, considered to be the Father of Medicine, reportedly said, “all disease begins in the gut. ” It took doctors centuries to start studying the connection between our gut microbiota and our health, which is now well-supported by research. We can improve the health and diversity of our gut microbiota with active strategies, such as exercising regularly, eating a healthy diet, and getting enough sleep. We might also be able to improve our gut microbiota by simply consuming live “good” bacteria and microorganisms called probiotics. Probiotics are now sold as capsules, pills, and powders, added to fruit juices, granola bars, and candy, and even put into cosmetics. Fermented foods, such as cultured yogurt, sauerkraut, and kombucha, contain them too. A cloud of hype and hope surrounds probiotics, which have been touted as a treatment (and means of prevention) for everything from intestinal disorders, mood and anxiety disorders, belly fat and obesity-related disorders. 

A recent article in the New York Times suggested that a probiotic of heat-killed bacteria can treat metabolic disorders like obesity. This suggestion was based on a study in Nature Medicine.The  New York Times journalist summarized the study as follows:

Researchers tested the bacterium, Akkermansia muciniphila, in 32 men and women who met the criteria for metabolic syndrome by having at least three of five conditions: high fasting blood sugar, high blood pressure, high triglycerides, low HDL (the “good” cholesterol) or excessive waist circumference.

…In a three-month trial, volunteers were randomized to one of three groups: daily tablets containing live bacteria, pasteurized bacteria [A. muciniphila] or a placebo. Compared with the placebo group, those who took pasteurized A. muciniphila had significantly improved insulin sensitivity and total cholesterol, and decreases in several blood markers of inflammation and liver dysfunction. They also had decreased body weight, fat mass and waist circumference, though those differences were not statistically significant. The live bacteria were largely ineffective.

1. This study was an experiment. What is the independent variable (IV) and what are its levels? What are the dependent variables (DV)? What do you think of the outcome measures chosen for this study?
   
   
2. How was the independent variable manipulated:  between or within groups? Which of the four basic experiments is it probably an example of: Pretest/postttest design? Posttest-only? Repeated measures? or Concurrent measures?
   
   
3. Can the results of this study be used to support the causal claim that “Taking pasteurized bacteria can improve people’s insulin sensitivity and total cholesterol?” Why or why not? (Apply the three criteria for causality.)
4. Look at the sample size of this study; do you think it’s a problem? Why or why not? Which of the four big validities does the sample size best fit into? 
5. Do the results above support the claim that probiotics like A. muciniphila can treat obesity? Why or why not? (Is internal validity relevant here? What about construct validity?) Can you conclude that probiotics can help you lose weight? Why or why not?
6. The New York Times article critically failed to mention that the researchers who conducted this study have a patent on the probiotic treatment and even started a company based on it. What do you think about this potential conflict of interest? 
7. Do you think it matters that this study’s somewhat promising findings were only with dead bacteria, whereas probiotics are generally defined as being live bacteria? Why or why not? (Which of the four big validities is being asked about in this question, if any?)
   
   

You might agree that the probiotic study described by this journalist has a few serious problems. For an example of probiotic research done right, check out this NPR news story on a current clinical trial examining if a specific probiotic could eliminate specific GI problems in children on antibiotics (courtesy of Dr. Jess Hartnett’s statistics blog).

Does this punctuation make me sound angry, or what.

Which of these two text messages seems the most sincere? Photo: Travis Carr

If you're an American young adult, you probably already follow certain unwritten rules about about punctuating text messages. FOR EXAMPLE, EMAILS AND TEXTS WRITTEN IN ALL CAPS ARE CONSIDERED A FORM OF SHOUTING. Text messages follow their own set of norms, and the punctuation we use can lead to different interpretations of the same words. Some linguistics researchers have studied the impact of different text-message punctuation. 

This brief article in Quartz summarizes the results of some of this research. The journalist introduced the research with this: 

Though periods can still signal the end of a sentence in a text message, many users will omit them (especially if the message is only one sentence long). This tendency now subtly influences how we interpret them.

First example

This study is an example of a 2x2 factorial design. Here's how the journalist summarized the study: 

In 2016, psychologist Danielle Gunraj tested how people perceived one-sentence text messages that used a period at the end of the sentence. Participants thought these text messages were more insincere than those that didn’t have a period. But when the researchers then tested the same messages in handwritten notes, they found that the use of a period didn’t influence how the messages were perceived.

a) What are the two independent variables (IVs) in this design? What are their levels?

b) What was the dependent variable? 

c) Do you think the IVs were manipulated as independent groups or within groups? 

d) Sketch a graph of the results of the study. Remember to put the DV on the y-axis. Put one IV on the x-axis and represent the other IV with two different lines or bar colors. 

 

Second example

For practice with a simple experiment (not a factorial), compare these two text messages. Which one do you think participants found to be more sincere?

JACKIE I AM SO SO SO SORRY! I thought you were behind us in the cab and then I saw you weren’t!!!!! I feel soooooooo bad! Catch another cab and ill pay for it for youuuuu

Jackie, I am so sorry. I thought you were behind us in the cab and then I saw you weren’t. I feel so bad! Catch another cab and I’ll pay for it for you.

e) These two messages were studied by linguist Deborah Tannen. What would have been the independent and dependent variables in the study that compared these two messages? How might you design the study--would a posttest only design be better, or a concurrent-measures design, and why? 

f) How might the results of this study change if they were conducted on an older sample compared to a younger one? 

Observing how people talk to their dogs

What's she saying to her dog? What category would her statement fall into?
Photo Credit: Shift Drive/Shutterstock

This entertaining article quotes several cute examples of how people talk to their dogs while they are out in public. The author, Alexandra Horowitz, is a psychologist who studies dog cognition. She has been recording the snippets of owner-dog conversation she hears as she goes throughout her day. (Most of those doing the talking are the owners.) She shares examples such as "You’re so cute and so smart. And worth money! I could marry you" and "You’ve got a lot to learn! A lot to learn!” and "Don’t even think about it.”

The main body of the article shares the author's categorization system. After jotting down dozens of dog-directed human comments, the author was able to classify them into categories. One category, for example, she titled Cheering Squad, for comments in which the owner is congratulating the dog's behavior. She dubbed another category Mom Commentary, described as, "Eyes on the dog, she sees everything. And she’s gotta talk about it." Another is Perfectly Implausible Instructions, in which owners direct the dog to do impossible things. And there are two other categories in the article: "We've Discussed This" and "Rhetorical Realm".

Read the article to find examples of each of these categories. The author's system provides a clear example of the observational research practices from Chapter 6. The author observed a large set of behaviors. Then she developed a categorization system in order to simplify what she observed.

1. This piece is meant to be entertaining. But if the author were to formally study how people talk to their dogs, the next step would involve coming up with clear operational definitions of each of the three categories. Take a look at the article and try your hand at developing operational definitions of Cheering Squad, Mom Commentary, and Perfectly Implausible Instructions. Write clear instructions that would enable a coder to decide which comments go in only one category (and not in the others).
   
   
2. Test out your operational definitions in the real world. Go out and find some people with dogs, and record some of your own snippets of conversation. Classify each snippet into the categories using your operational definitions, keeping in mind you'll probably need an "other" category. 
   
   
3. A good categorization system requires inter-rater reliability. How would you establish inter-rater reliability for your system? (Hint: You'll need to involve another rater who uses the same categories to categorize the same pieces of conversation.
   
   
4. Any researcher will tell you how difficult it is to develop a reliable coding system. What issues did you have when you tried this out yourself? 

Which Ice Cream Flavor Are You?

Many flavors, many personalities: How could we test whether an ice cream personality test is reliable and valid? Photo credit: Okanmetin/Getty Images

At the height of summer, we all start considering the same, timeless question: Can your personality can be accurately summarized by an ice cream flavor? This Buzzfeed poll asks you to answer a series of summer-activity-related questions and then delivers a verdict. Go take it yourself! I'll wait. 

The first time I took the poll, I answered the questions seriously and was told that I am characterized as "Coffee," along with this description: 

You’re a go-getter, a hard worker who doesn’t pause to reflect on what you have accomplished in the past because you are so focused on what you want to accomplish in the future. You seek to inspire the people around you and help to propel them to greater achievements. At the same time, you are approachable and kind rather than snotty and entitled. It’s basically the best of all worlds.

I sure felt good reading that personality description! The second time I took the quiz, I clicked on questions randomly haphazardly, just to see what other feedback there might be. This time I got "Chocolate", with this description:  

Mmm, you’ve got a truly rich and decadent side that you aren’t afraid to share. You make people feel good — really good. You’ve got a knack for excelling in social situations and know just how to be a crowd-pleaser. Keep doing your thing and living that luscious life.

Of course, nobody is going to mistake a Buzzfeed poll for a legitimate personality measure. But it does provide a fun way to consider issues raised in Chapters 5 and 6. Take the quiz yourself and then prepare these questions:

 

1.     What flavor ice cream did you get? What did the results say about you?

 

2.     Do you think this quiz is reliable? What kind of reliability is relevant for this measure? How would you empirically establish the quiz's reliability? Here's a hint: You'd start with a large sample of people and have them take this ice cream quiz, and then..... (what?)

3.  Now think about the measurement validity of this quiz. I'll narrow down the question for you here: Are people who score in the Coffee category actually more likely to be "hardworking," "inspiring," and "approachable," as the description suggests? How would you establish the criterion validity of this quiz? Here's a hint: You'd start with a large sample of people and have them take the ice cream quiz. You'd separate them into people who got the Coffee category and those who did not get Coffee. How should the Coffees be different from the non-coffees?

 

4.     What might we change about the quiz if we wanted to study ice cream preferences in a scientific way? 

 

5.  Read this description on the Barnum effect. Reflect on how the Barnum effect might be at play for people who take this Buzzfeed quiz. 

 

Many thanks for this idea to Josh Maxwell of the University of New Mexico!

 

Suggested answers to selected questions.

2. Test-retest reliability is the most relevant. To establish it, you'd have a large sample of people take the Buzzfeed quiz at one time point and record everyone's category (e.g., Coffee, Chocolate, etc.) Then you'd track down the same group of people again some time later (maybe 4 weeks later) and have them take the same quiz again. People who were scored Coffee at Time 1 should also score Coffee at Time 2, and people who were scored as Chocolate at Time 1 should also score as Chocolate at Time 2. (You'd compute the correlation between these two category assignments using Cohen's Kappa.)

3. To establish this criterion validity, you'd recruit a large sample of people and have them take the quiz, recording those who got Coffee and those who did not. Then you'd need to measure variables such as "hardworking" "inspiring" and "approachable." I'm not sure how to measure "inspiring" or "hardworking," but you could probably measure "approachable" using the Big Five trait of Agreeableness (here's one measure of that). If the Buzzfeed poll is valid, then a simple bar graph (and an independent groups t-test) should show that people who are assigned Coffee should score higher on agreeableness than people who were assigned some other flavor.