There's a new piece in the Atlantic summarizing some research on phone calls. The journalist's piece is tantalizing, but it doesn't provide some of the information you need to evaluate the research behind it.
According to the article, a data analytics firm analyzed a bunch of recorded phone calls--about 4 million of them--from customer service interactions. According to the journalist, the firm counted how fast people talked, how many words they used, and even how long they would wait on hold before hanging up! The journalist reported:
In some sense, [the] findings hew to cultural stereotypes. The fast-talkers are concentrated in the North; the slow-talkers are concentrated in the South.
Specifically, the fastest talkers were found in Oregon, Minnesota, Massachusetts, Kansas, and Iowa. The slowest were in North Carolina, Alabama, South Carolina, Louisiana, and Mississippi.
The Atlantic piece includes some colorful maps of the USA, indicating the fast, slow, and medium-talking states. Check them out!
a) While you're there, see if you can figure out how they defined "speed of talking" in this research. Also see if you can find out how much faster the Oregonians talk, compared to the Mississippians (how big is the effect? Is it statistically significant?). Before playing up these regional differences in a magazine article, this seems like important stuff to know, right?
The Atlantic piece goes on to discuss how "wordy" the phone calls were. Here the results were different. The journalist explains:
...speedy speech doesn’t necessarily equate to dense speech. [The analytic form] also used its dataset to analyze the wordiest speakers, state by state—the callers who, regardless of their tempo, used the most words during their interactions with customer service agents.
Some of the slower-talking states (Texas, New Mexico, Virginia, etc.) are also some of the wordier, suggesting a premium on connection over efficiency. Some of the fastest-talking states (Idaho, Wyoming, New Hampshire) are also some of the least talkative, suggesting the get-down-to-business mentality commonly associated with those states.
b) Can you find out, from the article, how much wordier the "wordy" states are compared to the less wordy states? How big are the effects?
c) And can you figure out from the Atlantic article exactly how the researchers operationalized "being wordy" and "talking fast?" How are these two variables different, exactly? (I couldn't find it from the article--let me know if you can!)
I'm being critical of the journalist's coverage here, but I'm also keeping in mind that it might not be her fault. The source of the data is not a peer-reviewed journal article (which would have been required to present this information). Instead, the source was a report by the for-profit analytics firm that had collected the data. Perhaps this firm did not report enough methodological and statistical details to allow careful reporting by the journalist.
However, I checked the website of the analytic firm, There, you can find the information you need. Their report mentions exactly how the two variables were operationalized and gave effect sizes of the results.
Specifically, you learn that talking speed was operationalized as "words per minute". And the differences between states were described this way:
For every 5 words a slow talking state utters, a fast talking state will utter 6.
You also learn that wordiness was operationalized as "total words in a phone conversation." and:
How big is the difference? A New Yorker will use 62% more words than someone from Iowa to have the same conversation with a business,
d) What do you think of these operationalizations of talking speed and wordiness? And what do you think of the magnitudes of the effects? Are these large differences--worthy of a news story on differences between states? If you were the journalist, would you have incorporated this information, or not?
You might also be interested in a second Atlantic story about which states are the "sweariest." Compared to the story on talking fast (and being wordy), this story is more informative. For one, it describes how swearing was operationalized. They
...examin[ed] more than 600,000 phone calls from the past 12 months—calls placed by consumers to businesses across 30 different industries. It then used call mining technology to isolate the curses therein, cross-referencing them against the state the calls were placed from.
The journalist also opined, (probably correctly):
...cursing is also conveniently specific as a data set; you've got your f-bombs and your double hockey sticks and your bodily functions, and, factoring in their permutations, you're good to go. Plus, you don't need much sophisticated sentiment analysis to ensure that your data are accurate: An f-bomb is pretty much an f-bomb, regardless of the contextual subtleties.
In the swearing story, the journalist also described the data better, explaining how the rates of swearing in the highest-swearing state and the lowest state compare:
People in Ohio cursed the most as compared to every other state in the Union: They swore in one out of about every 150 phone conversations. Ohio was followed, respectively, by Maryland, New Jersey, Louisiana, and Illinois.
And who swore the least? Washingtonians. They cursed, on average, during one out of every 300 conversations. (Yes, this means that Ohioans swear at more than twice the rate of Washingtonians....)
d) Now you have a better picture of the data. What do you think of the sizes of these effects? Big or small? Important, or not?
e) The headline of this story announces that Ohio "is the sweariest state in the Union." What do you think of that headline? Specifically, does that headline overstate the results? Does it overgeneralize from the measures they used?