Everyday Research Methods

Tragedy in Colorado spurs gun sales

In the news recently: Applications for gun safety (which are required in order to purchase a gun in some U.S. states) have increased about 25% recently.  The timing of the increase comes right after the tragedy in Aurora, CO, in which a gunman killed 12 people and injured dozens of others.

Here's an example of a story about gun sales.

The story reports that gun sales have increased not just in Colorado, but also in other U.S. states.

These data might remind you of the quasi-experimental study in Chapter 12 in which larceny rates increased following the introduction of television in several U.S. cities.

a) Sketch a graph of the pattern of gun sales reported in this news story. What kind of quasi-experimental study is illustrated here?

b) There are two possible causal explanations for the increase in gun sales. One explanation proposes that after a gun tragedy like this, people want to buy guns because they want to protect themselves. Another explanation proposes that people are buying guns because they anticipate stricter gun laws in the future. Can you think of some kind of quasi-experimental data that would help to distinguish between these two causal explanations?

Do magazine personality tests tell us anything?

Have you ever seen those magazine articles entitled, "What does your _____ say about you?" Magazines like to make us think we can use people's pets, music preferences, Saturday wardrobe, or cars as secret keys to making personality judgments about them. Is that really possible?

Here's an example of this kind of article, from Men's Health, claiming to be able to tell you "What your drink order says about you."

Of course, magazine articles like these aren't meant to be scientifically rigorous--they are mainly for entertainment. Nevertheless, what is this particular article able to say? Use your research methods skills to interrogate this piece.

a. Read the first three paragraphs--what sources of evidence does the author use for his claims? 

b. Read the fourth paragraph--the one about the personalities of people who order light beers. The paragraph states,

Domestic Light Beer (Bud Light, Miller Light)

You’re impulsive and quick to take risks. You’re respectful of authority, accepting of most people, and generally easy to get along with.

Can you turn the above statements into an association claim? (Hint--identify two variables at a time.)

c. Another part of this article states that people who order vodka "care about fashion and appearance." How might a study be conducted that would be able to test whether people who order vodka are more fashion-conscious?  Make a graph of data that would support this claim.

 

Suggested answers

a. It seems that the authors used three sources of data. One was a published study conducted by the University of Pittsburgh; this study is mentioned in the first sentence but its findings are not discussed further in this article. Another source of data was market research reports which had been produced by some market research companies. A third source was anecdotal--that is, experiential evidence--gathered from bartenders. (You might pause to think about the credibility and accuracy of these three types of data, using guidelines from Chapter 2.)

b. This paragraph is describing bivariate assocations between the variable, "preference for light beer" and various personality traits. One association claim you could make is, "preference for light beer is associated with impulsivity." Another could be, "preference for light beer is associated with risk taking," and so on.

c.  In order to show that people who order vodka "care about fashion and appearance," researchers would select a sample of people and measure both their tendency to order vodka and their degree of interest in fashion and appearance. Then they would see whether the two variables are correlated in the sample. Picture the correlation as a scatterplot with "tendency to order vodka" on one axis and "degree of interest in fashion and appearance" on the other axis. The dots (each representing one person) would arrange themselves in a cloud of points sloping upward, representing a positive association between these two variables.

Follow up question--using the answer to Question c, decide how you could test the association claims made in Question b. How could you graph these claims?

 

Ethics: New examples of fraud in psychology

Two more reports of research fraud have surfaced in the subfield of social psychology. You can read about two new suspected cases in this news piece by the journal Nature.

Read the article and think about these questions:

a) Are the cases described here examples of data falisification or data fabrication? (Is this possible to determine from the report?)

b) How did the UPenn researcher, Dr. Uri Simonsohn, use statistical techniques to detect suspicious data? 

c) What does this mean for the reputation of social psychology ? Will the public come to mistrust psychological research?  What are the costs and benefits to Dr. Simonsohn's investigation of others' data?

This blog has covered an earlier example of research fraud. You can read that entry here.

 

Links between spanking and later mental health

A recent article in Yahoo! news network describes research on spanking and mental health.  The journalist reports:

Corporal punishment was associated with increased odds of anxiety and mood disorders, including major depression, panic disorder, post-traumatic  stress disorder, agoraphobia and social phobia. Several personality  disorders and alcohol and drug abuse were also linked to physical  punishment, the researchers found.

The study, originally published in the journal Pediatrics, intended to test the effects of physical punishment separately from more severe physical or sexual abuse. The journalist reported that the method involved

about 34,000  individuals aged 20 or older gathered from the U.S. National Epidemiologic  Survey on Alcohol and Related Conditions. Participants were questioned face-to-face and asked, on a scale of "never" to "very often," how often  they were ever pushed, grabbed, shoved, slapped or hit by their parents or another adult living [in] their home.

a. The journalist's story mentions that while the study shows that physical punishment is associated with some mental illnesses, the results cannot "prove that one causes the other." Why not?  What are three possible causal paths for this association? (Hint:  Think about both temporal precedence and the third variable problem)

Part of the story reports that

The researchers adjusted the data to take into account  socio-demographic factors and any family history of dysfunction.

b. Was family history of dysfunction one of your third variables in question a? If so, what does the above statement mean?

One of the experts who was interviewed about this study said,

"While it's a well-done study, looking at a national data sample, there are limitations in the way the study was done," said Dr. Andrew Adesman, chief of developmental and behavioral pediatrics at Steven and Alexandra Cohen Children's Medical Center in New Hyde Park, N.Y. "There are  limitations to relying on adults recalling childhood experiences...."

c.  This expert is evaluating two of the big validities. Which of his quoted statements evaluate which validities, and why?

Suggested answers

a. The two variables in this association are a history of physical punishment and a diagnosis of mental illness. It could be the case that physical punishment causes mental illness. It could also be the case that kids who show early signs of mental illness (somehow) cause their parents to be more physical with them. Or, it is possible that a third variable, such as family dysfunction or poverty, is associated with both mental illness and physical punishment.

b. The statement means that the researchers used a multivariate correlational design, testing whether there is still a link between spanking and mental illness even when controlling for family dysfunction.  Since the link is still there, it means that family dysfunction can actually be ruled out as a third variable explaining the relationship.

c. The expert praised the external validity of the study when he said, "While it's a well-done study, looking at a national data sample".  But he criticized the construct validity of the study when he said, "there are limitations to relying on adults recalling childhood experiences". By mentioning that people may not accurately recall their childhoods, this expert is suggesting that the study's operationalization of spanking may have been flawed.