Everyday Research Methods

How fast do people talk? Depends on where you live....

How fast do people talk? How wordy are they? Do they swear a lot? Photo credit: Eugenio Marongiu/Shutterstock

There's a new piece in the Atlantic summarizing some research on phone calls. The journalist's piece is tantalizing, but it doesn't provide some of the information you need to evaluate the research behind it.

According to the article, a data analytics firm analyzed a bunch of recorded phone calls--about 4 million of them--from customer service interactions. According to the journalist, the firm counted how fast people talked, how many words they used, and even how long they would wait on hold before hanging up! The journalist reported:

In some sense, [the] findings hew to cultural stereotypes. The fast-talkers are concentrated in the North; the slow-talkers are concentrated in the South.

Specifically, the fastest talkers were found in Oregon, Minnesota, Massachusetts, Kansas, and Iowa. The slowest were in North Carolina, Alabama, South Carolina, Louisiana, and Mississippi.

The Atlantic piece includes some colorful maps of the USA, indicating the fast, slow, and medium-talking states. Check them out!

a) While you're there, see if you can figure out how they defined "speed of talking" in this research. Also see if you can find out how much faster the Oregonians talk, compared to the Mississippians (how big is the effect? Is it statistically significant?). Before playing up these regional differences in a magazine article, this seems like important stuff to know, right?

The Atlantic piece goes on to discuss how "wordy" the phone calls were. Here the results were different. The journalist explains:

...speedy speech doesn’t necessarily equate to dense speech. [The analytic form] also used its dataset to analyze the wordiest speakers, state by state—the callers who, regardless of their tempo, used the most words during their interactions with customer service agents.

and also:

Some of the slower-talking states (Texas, New Mexico, Virginia, etc.) are also some of the wordier, suggesting a premium on connection over efficiency. Some of the fastest-talking states (Idaho, Wyoming, New Hampshire) are also some of the least talkative, suggesting the get-down-to-business mentality commonly associated with those states.

b) Can you find out, from the article, how much wordier the "wordy" states are compared to the less wordy states? How big are the effects? 

c) And can you figure out from the Atlantic article exactly how the researchers operationalized "being wordy" and "talking fast?"  How are these two variables different, exactly? (I couldn't find it from the article--let me know if you can!)

I'm being critical of the journalist's coverage here, but I'm also keeping in mind that it might not be her fault. The source of the data is not a peer-reviewed journal article (which would have been required to present this information). Instead, the source was a report by the for-profit analytics firm that had collected the data. Perhaps this firm did not report enough methodological and statistical details to allow careful reporting by the journalist.

However, I checked the website of the analytic firm, There, you can find the information you need. Their report mentions exactly how the two variables were operationalized and gave effect sizes of the results.

Specifically, you learn that talking speed was operationalized as "words per minute". And the differences between states were described this way:

For every 5 words a slow talking state utters, a fast talking state will utter 6.

You also learn that wordiness was operationalized as "total words in a phone conversation." and: 

How big is the difference? A New Yorker will use 62% more words than someone from Iowa to have the same conversation with a business,

d) What do you think of these operationalizations of talking speed and wordiness? And what do you think of the magnitudes of the effects? Are these large differences--worthy of a news story on differences between states? If you were the journalist, would you have incorporated this information, or not?

You might also be interested in a second Atlantic story about which states are the "sweariest." Compared to the story on talking fast (and being wordy), this story is more informative. For one, it describes how swearing was operationalized. They

...examin[ed] more than 600,000 phone calls from the past 12 months—calls placed by consumers to businesses across 30 different industries. It then used call mining technology to isolate the curses therein, cross-referencing them against the state the calls were placed from.

The journalist also opined, (probably correctly):

...cursing is also conveniently specific as a data set; you've got your f-bombs and your double hockey sticks and your bodily functions, and, factoring in their permutations, you're good to go. Plus, you don't need much sophisticated sentiment analysis to ensure that your data are accurate: An f-bomb is pretty much an f-bomb, regardless of the contextual subtleties.

In the swearing story, the journalist also described the data better, explaining how the rates of swearing in the highest-swearing state and the lowest state compare:

People in Ohio cursed the most as compared to every other state in the Union: They swore in one out of about every 150 phone conversations. Ohio was followed, respectively, by Maryland, New Jersey, Louisiana, and Illinois.

And who swore the least? Washingtonians. They cursed, on average, during one out of every 300 conversations. (Yes, this means that Ohioans swear at more than twice the rate of Washingtonians....)

d) Now you have a better picture of the data. What do you think of the sizes of these effects? Big or small? Important, or not?

e) The headline of this story announces that Ohio "is the sweariest state in the Union." What do you think of that headline? Specifically, does that headline overstate the results? Does it overgeneralize from the measures they used?

Playing with money makes kids act selfish

 

In the study, kids who handled coins were subsequently less helpful and generous, compared to kids who handled buttons. Photo credit:Andrey Arkusha/Shutterstock

What happens when people handle money? It turns out that many psychologists are interested in studying the psychological consequences of having money. Researchers have typically found that being wealthy, or feeling temporarily wealthy is associated with a number of self-interested outcomes, such as behaving selfishly, breaking laws, and being unethically (for example). Now there's a study that investigated the impact of money on kids. Does handling money have negative effects on kids, too?

 

Here's the link to the story in the Pacific Standard. According to the journalist, the researchers studied kids who were between 4 and 6 years of age. The journalist mentions that the article reported five studies. Let's examine the first one, as described by the journalist:

One of them featured 64 Polish children, ages four to six. Participants began by sorting 30 objects: either coins (by denomination), or similarly sized buttons (by color).

"Next, the children moved to another room, where they met a new experimenter who asked for help readying a task for another child," the researchers write."She asked the children to bring her as many red crayons as they could from a box in the far corner of the room." She then handed them baskets in which they could place the crayons.

"The children who had sorted money brought the experimenter fewer crayons than did the children who sorted buttons," the researchers report.

a) What are the main two variables in this study? For variable, decide if it is manipulated or measured.

b) What kind of study do you think is being conducted here? A correlational study or an experiment? Explain your answer.  If it's an experiment, which of the four basic designs is it?

c) Sketch a graph of the results they describe.

d) Can the research support the journalist's headline, which was "Handling Money Decreases Helpful Behavior Among Young Children"? Apply the three criteria for causation.

e) You could reflect on the construct validity of this study. How would you ask construct validity questions about the two variables studied here?

It's worth pointing out that the journalist mentions:

Interestingly, this held true even for children who could not identify which coins were more valuable (that is, which of them "would buy the most candy").

This statement is about a potential moderator: Did kids' knowledge of money change the relationship between playing with money and their subsequent helpfulness? Apparently not: Knowledge of money is not a moderator.

Here's another experiment summarized by the journalist:

In another experiment, children who spent more time sorting money, "and hence were exposed to it longer, took more stickers for themselves, and donated fewer stickers to other children." In contrast, the amount of time they spent sorting buttons or candy did not produce this effect.

f) What about this study? The journalist called this an experiment. But was it? What do you think?

g) What seem to have been the variables in this study?

 

 

Suggested answers

a) The main two variables are 1) what kids played with (money or buttons)--this was a manipulated variable. and 2) how many crayons kids brought over--this was a measured variable.

b) This was an experiment (assuming they randomly assigned kids to the money or button conditions), because the variable of "what kids played with" was manipulated.

In fact, this is a post-test only experiment. It has an independent groups design.

c) A bar graph would have "number of red crayons" on the y axis, and the x-axis would have a label under two bars: "money" and "buttons."  The "buttons" bar should be higher than the "money" bar.

d) This experiment is consistent with that causal claim. First, the results support covariance--there is a relationship between playing with money and being less helpful. Second, the experimental method ensured that kids played with the money (vs. buttons) before they had the opportunity to help the experimenter. Third, the experimental nature of the study helps with internal validity. The journalist doesn't say, but if we assume kids were randomly assigned to play with money vs. buttons, this controls for many individual difference variables such as the age of the kids, their preexisting levels of helpfulness, their comfort in this experimental situation, and so on. Both tasks (sorting money and buttons) involved objects that were about the same size, and involved a categorization task, so that's not a confound. It does seem to support the claim.

e) To assess construct validity, we'd ask, first, "how well does sorting coins (vs. buttons) manipulate the variable, 'playing with money'?" Then we'd ask, "how well does bringing an experimenter red crayons measure the variable, 'helpfulness'?"

f) This one is more difficult to classify as experimental or correlational. It's not clear if the variable, "amount of time spent with money" was a manipulated variable or a measured variable. It could have been manipulated, because you could assign some kids to play with money (or buttons) for a short time or a long time. But it might also be measured, because you could simply observe how long kids choose to play with some money or some buttons. This is an example of a time we'd have to go to the original journal article to learn more about the study. The journalist left out some details.

g) One variable is "amount of time being spent." Another variable is "how many stickers children gave to other kids". And I think another variable is "playing with buttons versus money."