Everyday Research Methods

Observing how people talk to their dogs

What's she saying to her dog? What category would her statement fall into?
Photo Credit: Shift Drive/Shutterstock

This entertaining article quotes several cute examples of how people talk to their dogs while they are out in public. The author, Alexandra Horowitz, is a psychologist who studies dog cognition. She has been recording the snippets of owner-dog conversation she hears as she goes throughout her day. (Most of those doing the talking are the owners.) She shares examples such as "You’re so cute and so smart. And worth money! I could marry you" and "You’ve got a lot to learn! A lot to learn!” and "Don’t even think about it.”

The main body of the article shares the author's categorization system. After jotting down dozens of dog-directed human comments, the author was able to classify them into categories. One category, for example, she titled Cheering Squad, for comments in which the owner is congratulating the dog's behavior. She dubbed another category Mom Commentary, described as, "Eyes on the dog, she sees everything. And she’s gotta talk about it." Another is Perfectly Implausible Instructions, in which owners direct the dog to do impossible things. And there are two other categories in the article: "We've Discussed This" and "Rhetorical Realm".

Read the article to find examples of each of these categories. The author's system provides a clear example of the observational research practices from Chapter 6. The author observed a large set of behaviors. Then she developed a categorization system in order to simplify what she observed.

1. This piece is meant to be entertaining. But if the author were to formally study how people talk to their dogs, the next step would involve coming up with clear operational definitions of each of the three categories. Take a look at the article and try your hand at developing operational definitions of Cheering Squad, Mom Commentary, and Perfectly Implausible Instructions. Write clear instructions that would enable a coder to decide which comments go in only one category (and not in the others).
   
   
2. Test out your operational definitions in the real world. Go out and find some people with dogs, and record some of your own snippets of conversation. Classify each snippet into the categories using your operational definitions, keeping in mind you'll probably need an "other" category. 
   
   
3. A good categorization system requires inter-rater reliability. How would you establish inter-rater reliability for your system? (Hint: You'll need to involve another rater who uses the same categories to categorize the same pieces of conversation.
   
   
4. Any researcher will tell you how difficult it is to develop a reliable coding system. What issues did you have when you tried this out yourself? 

Which Ice Cream Flavor Are You?

Many flavors, many personalities: How could we test whether an ice cream personality test is reliable and valid? Photo credit: Okanmetin/Getty Images

At the height of summer, we all start considering the same, timeless question: Can your personality can be accurately summarized by an ice cream flavor? This Buzzfeed poll asks you to answer a series of summer-activity-related questions and then delivers a verdict. Go take it yourself! I'll wait. 

The first time I took the poll, I answered the questions seriously and was told that I am characterized as "Coffee," along with this description: 

You’re a go-getter, a hard worker who doesn’t pause to reflect on what you have accomplished in the past because you are so focused on what you want to accomplish in the future. You seek to inspire the people around you and help to propel them to greater achievements. At the same time, you are approachable and kind rather than snotty and entitled. It’s basically the best of all worlds.

I sure felt good reading that personality description! The second time I took the quiz, I clicked on questions randomly haphazardly, just to see what other feedback there might be. This time I got "Chocolate", with this description:  

Mmm, you’ve got a truly rich and decadent side that you aren’t afraid to share. You make people feel good — really good. You’ve got a knack for excelling in social situations and know just how to be a crowd-pleaser. Keep doing your thing and living that luscious life.

Of course, nobody is going to mistake a Buzzfeed poll for a legitimate personality measure. But it does provide a fun way to consider issues raised in Chapters 5 and 6. Take the quiz yourself and then prepare these questions:

 

1.     What flavor ice cream did you get? What did the results say about you?

 

2.     Do you think this quiz is reliable? What kind of reliability is relevant for this measure? How would you empirically establish the quiz's reliability? Here's a hint: You'd start with a large sample of people and have them take this ice cream quiz, and then..... (what?)

3.  Now think about the measurement validity of this quiz. I'll narrow down the question for you here: Are people who score in the Coffee category actually more likely to be "hardworking," "inspiring," and "approachable," as the description suggests? How would you establish the criterion validity of this quiz? Here's a hint: You'd start with a large sample of people and have them take the ice cream quiz. You'd separate them into people who got the Coffee category and those who did not get Coffee. How should the Coffees be different from the non-coffees?

 

4.     What might we change about the quiz if we wanted to study ice cream preferences in a scientific way? 

 

5.  Read this description on the Barnum effect. Reflect on how the Barnum effect might be at play for people who take this Buzzfeed quiz. 

 

Many thanks for this idea to Josh Maxwell of the University of New Mexico!

 

Suggested answers to selected questions.

2. Test-retest reliability is the most relevant. To establish it, you'd have a large sample of people take the Buzzfeed quiz at one time point and record everyone's category (e.g., Coffee, Chocolate, etc.) Then you'd track down the same group of people again some time later (maybe 4 weeks later) and have them take the same quiz again. People who were scored Coffee at Time 1 should also score Coffee at Time 2, and people who were scored as Chocolate at Time 1 should also score as Chocolate at Time 2. (You'd compute the correlation between these two category assignments using Cohen's Kappa.)

3. To establish this criterion validity, you'd recruit a large sample of people and have them take the quiz, recording those who got Coffee and those who did not. Then you'd need to measure variables such as "hardworking" "inspiring" and "approachable." I'm not sure how to measure "inspiring" or "hardworking," but you could probably measure "approachable" using the Big Five trait of Agreeableness (here's one measure of that). If the Buzzfeed poll is valid, then a simple bar graph (and an independent groups t-test) should show that people who are assigned Coffee should score higher on agreeableness than people who were assigned some other flavor.