Ethan Wragge, a senior at Creighton University, recently sank seven shots in a row in a college basketball game against Villanova. Wragge appeared to have a "hot hand"--it seemed as if he couldn't miss a shot.
But is the "hot hand" a real phenomenon in basketball? Operationally speaking, does making a couple of shots in a row in a basketball game make it more likely that a player will sink the third one? Or the fourth one?
This is a story about replication, effect size, and sample size, as well as creativity in research. Thirty years ago, social psychologists Gilovich, Vallone, and Tversky studied the jump shots of the Philadelphia 76ers, the free throws of the Boston Celtics, and Cornell University students, and concluded that the "hot hand" is a myth. Their data suggested that players are not actually more likely to make their next shot if they'd sunk the ones before. The Wall Street Journal reports:
The reason the authors chose the 76ers as their guinea pigs was far from scientific. NBA statistics were so primitive at the time, Gilovich said, that the 76ers were the test subjects by default: They were the only team that kept useful records of their jump shots.
Times are different in 2014. Teams collect more data about their players. A lot more data:
....statheads say they are now benefiting from a numbers glut. The data driving the latest hot-hand paper comes from the high-resolution, missile-tracking cameras recently installed in every NBA arena that log location coordinates of each player multiple times per second. This essentially turns an ordinary game into a trove of data.
In fact, three Harvard University students analyzed data that had been collected on 70,000 basketball shots. A database of 70,000 basketball shots is much more likely to find a "hot hand" effect than the database from just one team.
Indeed, when the Harvard students analyzed the data, they concluded that
a player can be more likely, not less likely, to make his next shot if he has made several in row. Their hot-hand estimate ranges from a 1.2 to a 2.4 percentage-point increase in likelihood.
This tale illustrates two research methods lessons.
First, in statistical terms, a large sample of 70,000 has a lot more power to detect patterns than a smaller sample. The large sample is like a strong, powerful flashlight--it can find anything. A smaller sample is like a weak candle--you might miss the small stuff.
In other words, you might say that the "hot hand" was there all along--it's just that earlier studies were not powerful enough (did not have enough shots to analyze) to detect it.
Second, we can consider effect size here. The downside, if any, to having a very large, powerful sample, is that even small effect sizes will be statistically significant. And indeed, this article describes a 1.2 to 2.4 percent increase in shooting percentages--is that a large effect, in your opinion?
In other words, this study had a lot of power to detect the hot hand--and they seem to have found it. But it does not seem to have very large impact (effect size) on whether the "hot" player is going to make the shot. In other words, if a player sinks, on average, 55% of his or her shots, the hot hand might increase that to 56 or 57%. Is that worth, as Wragge says, betting on?
Wragge himself said he is a believer. "It's like the automatic, unconscious feeling," he said. "If I hit two in a row, I bet the third one almost always goes in."
Finally, it's important to note that this data set is not only larger, it's also richer. The story reports that the researchers not only analyzed whether a player made a shot. They also analyzed what type of shot it was. Some shots are more difficult than others--and players who feel "hot" are actually more likely to try for more difficult ones. According to the article:
By separating easier shots like open layups from difficult shots like contested three-pointers, they say, they fixed a problem that may have skewed past studies.
Bigger data sets with more information allow researchers to control for extra sources of variability. In this case, by controlling for type of shots in the data, the researchers were able to identify, and control for, noise in their data--making it even more likely they could find the hot hand they were looking for.
