Did you watch the Netflix limited series Queen's Gambit, which came out in the fall of 2020? Many people did; it was one of Netflix's most popular series to date. Many folks have speculated that the release of this show influenced people to play chess for the first time, or to reignite an old chess interest they once had.
Here is a blog post by a data scientist David Zhang, who was able to analyze traffic on the website, chess.com, over the past several months. He analyzed the number of new users who started new accounts on the site. Chess.com is the most popular chess community on the Internet.
Zhang graphed the number of games played by United Kingdom (UK) players by day from July 2020 (three months before the Queen's Gambit was released) to March 2021 (five months after). The Netflix show appeared in October, 2020. Please click on this story link and scroll to the first graph that you see, which appears under the heading, Number of new players.
Questions
a) This graph, by itself, should remind you of one of the four quasi-experimental designs in Chapter 13. Which one?
b) What is the quasi-independent variable in this study? What are its levels?
c) What is the dependent variable in this study?
d) When we analyze how well a quasi experiment can support a causal claim (such as "Queens Gambit caused people to join chess.com"), we need to consider the design and the results. What do you think--do the design and results show here help us rule out any internal validity threats?
e) Many folks would nominate history threat as a likely issue for this situation. That is, they might suggest that some other event, other than Queen's Gambit, occurred in October that could have caused chess to become more popular. What do you think--is that a reasonable critique? If so, what historical event would you nominate that would have occurred at the same time (October 2020), and would have affected most people in this UK sample?
(notice what the author of this post writes about a potential history threat:
....other chess events have occurred around the time frame analysed. For example, the first PogChamps tournament involving popular streamers was hosted in May 2020 by chess.com, which is likely to have spread awareness and garnered appeal for online chess.)
f) Other folks might say that maturation is a threat here. They might say that "perhaps people just naturally and spontaneously became more interested in chess over time." However, in my view, you can use the results to argue against this interpretation. You can discuss the stable baseline of chess interest before October 2020 as you prepare an answer to this critique.
Now scroll down the blog post a little further, to where the author has depicted chess interest in four countries--the UK, Italy, Colombia, and China. These countries released the Queen's Gambit as well, but at later dates than the UK; China did not air this show at all because it restricts foreign media.
g) This graph, by itself, should remind you of one of the four quasi-experimental designs in Chapter 13. Which one?
h) What are the quasi-independent variables (there are two) in this new study? What are their levels?
i) What is the dependent variable?
j) What do you think--how well do these results and design support the causal claim that "Queens Gambit caused people to join chess.com"?
You might be interested to read the author's analysis of the causality question, where he writes:
It is important to question the causal relationship between the two. It is likely that watching the show alone does not directly account for the entire increase in online chess popularity; many other factors may be at play. The current COVID situation has left many working from home, with some suggesting that online chess has profited as an outlet. Additionally, chess is by nature a multiplayer game and users lead to chess from The Queen’s Gambit may have propagated their newfound interest to others through their own social networks. Finally, other chess events have occurred around the time frame analysed. For example, the first PogChamps tournament involving popular streamers was hosted in May 2020 by chess.com, which is likely to have spread awareness and garnered appeal for online chess.
