Photo credit: LukaFunduk/Deposit Photos
Before you read on, take out a pencil and draw the Apple logo (no peeking at your own device!) Now try to draw the Adidas logo, too (no peeking at your feet, either).
Which one was easier? How confident are you in the accuracy of your drawings?
After you've drawn these two logos, visit this website, where you can compare your own drawings to those of other people. You can also see the official logos for Apple and Adidas. How well did you do? More importantly, how well did the people in their study do?
This website presents data from a study of Americans who were asked to draw a variety of brand logos. The website introduces the project:
...we asked over 150 Americans to draw 10 famous logos from memory as accurately as they could. Based on more than 1,500 drawings created over a period of 80 hours, the results reveal that, far from being stamped perfectly in our collective memory, these ubiquitous emblems largely exist as fuzzy visions in our mind's eye.
The text on the website reports:
Overall, 16 percent of people drew near perfect logos, and 37 percent were good but not perfect. As we would expect, the more complex the logo, the less likely people are to remember it in full.
Across the 10 brands (which included not just Apple and Adidas, but also Starbucks, Foot Locker, Burger King, and Domino's), the percentage of people who sketched near-perfect logos changed. For example, the website reports that 20% drew near-perfect Apple logos, whereas 12% drew near-perfect Adidas logos.
You can evaluate certain methodological aspects of this project. For example, here's a quote from their discussion of the Apple logo results:
Interestingly, a smaller proportion of people in our experiment put the bite on the wrong side of the apple (22 percent)
a) Wait, was this study really an experiment? why or why not?
The article made some frequency claims. For example,
One in 5 people thinks the Foot Locker referee wears a hat (he doesn't), and nearly half of people believe the Starbucks mermaid does not wear a crown (she does).
b) Why can each of these two claims be considered frequency claims?
c) When people make frequency claims, external validity (also known as generalizability) is pretty crucial. Think about the external validity of this project. Do you think the study they conducted is able to support the claim that "one in 5 people thinks the Foot Locker referee wears a hat?" What information do you need to know in order to evaluate the external validity of this claim?
d) Now consider statistical validity. How precise are the estimates on this website? The study estimated that 20% of the sample drew near-perfect Apple logos. But what's the precision of that estimate? In other words, what's the 95% CI, or margin of error? You can compute it here, if you know the sample size (which, just to remind you, was around 150).
What was the MoE you found? What would happen to that MoE if the sample had been larger? (Caveat--MoE computations actually depend on having a random sample, which we don't have, so these estimates are not going to be perfect.)
e) Finally, consider construct validity. The study needed to decide how accurate each person's sketch was. One of their footnotes reads, "accuracy based on independent ratings by a panel of marketing professionals." How do you think they obtained these ratings--what might their procedure have been?
f) Using the large matrices of drawings, make some comments about the construct validity of their operationalization of "accuracy".
Thanks to Marianne Lloyd for sending me this page!