Sometimes journalists or politicians call out research projects that they consider to be a waste of money, such as a line of research in biology and ecology in which shrimp ran on a treadmill. This news story from National Public Radio covers some examples of research whose measures or instruments have been criticized as silly, because they seem frivolous--at least on the surface.
According to the story,
The treadmills were just a small part of it, a way to measure how shrimp respond to changes in water quality. Burnett says the first treadmill was built by a colleague from scraps and was basically free, and the second was fancier and cost about $1,000. The senator's report was misleading, says Burnett, "and it suggests that much money was spent on seeing how long a shrimp can run on a treadmill, which was totally out of context."
In the language of the textbook, the behaviors of shrimp on a treadmill can be called an operationalization, in this case, as a way to operationalize the variable "reaction to water quality." On the surface, some research operationalizations can seem silly, but they almost always represent something more serious.
Rather than criticize how frivolous an operationalization is, we should ask how well that operationalization represents the construct it is meant to measure (that is, we should interrogate its construct validity).
The same story gives another example of a "silly study" that was attacked by some critics:
[One] group has also criticized other government-funded studies, including a National Institutes of Health grant for measuring nicotine exposure in toenail clippings. "They used recovery money, money that was meant to more or less stimulate the economy," says Lafferty. "Interesting use of money — mailing in toenail clippings."
That use of recovery funds was defended by Lawrence Tabak, deputy director of the National Institutes of Health. And he says the toenail study has an important goal that its critics didn't mention — trying to assess people's risk of lung cancer.
Consider the example above. What variable are the researchers operationalizing by measuring the qualities of toenail clipplings? What other measures, or operationalizations, could they also use to meausre the same construct? What evidence would support the construct validity of the toenail clipping measure?
Comments