After matching students based on their pre-college voting behaviors, the researchers found that students who had taken a political science class became more likely to register to vote. Photo credit: Lakshmiprasad/Depositphotos
In the online news outlet called The Conversation, researchers write summaries of their own research studies for a general audience.
Here's an example from political scientists Frank Fernandez and Matthew Capaldi, both of the University of Florida. They introduce their study by writing,
...improving college student voter turnout is a national issue....According to data from the National Study of Learning, Voting, and Engagement, about 1 in 4 students – including at both two- and four-year colleges – were not registered to vote in the 2016 or 2018 elections.
The researchers conducted a study of 2,000 students at community colleges. All the students had completed an extensive survey about their civic behaviors both before and during college. Students also reported the classes they had taken in college.
The researchers used a "propensity score matching procedure", which is similar in some ways to multiple regression. Specifically, they matched up students who had the same demographic characteristics (e.g., race, parental education, family income, gender). They also matched students on whether, before going to college, they had been registered to vote or voted in elections. The researchers were mainly predicting whether students had newly registered to vote (since starting college) as well as whether they had actually voted since starting college. The authors write about their results:
After taking students’ prior civic engagement and other college experiences into account, we found that students who took at least one political science course were 9% more likely to register to vote than those who did not.
Additionally, we found that students who took at least one political science class were 8% more likely to vote.
Questions
a) Name the variables in this study--there are at least seven variables mentioned in the summary provided above.
b) Is this a correlational study or an experiment? Explain your answer.
c) The authors imply that taking a political science course seems to cause students to be more likely to register to vote and to actually vote. Let's apply the three criteria for causation to this statement. First, do the results support covariance?
d) Second, does the method of the study clearly establish temporal precedence? Why or why not?
e) Finally, the authors used the matching technique to control for several alternative explanations such as race, parental education, family income, gender, as well as whether, before going to college, they had been registered to vote or had voted in elections. Can you think of any additional alternative explanations that they did not control for?
f) In their summary in The Conversation, the authors discuss some of the downsides of their methodology. For example, they write, "We relied on self-reported data, so there is no practical way to confirm that they registered to vote or turned out to vote." This critique is focused on construct validity--specifically, criterion validity. That is, they are admitting that they don't know whether or not a self-report item about voting actually predicts the behavioral criterion of voting. How could you run a pilot study to confirm that people who say they voted in a certain election have actually voted (and vice versa--people who say they did not vote have actually not voted)?
Here's a link to their study , which was published in Educational Researcher (sorry--paywalled).
Suggested answers:
a) The variables include race, parental education, family income, gender, being registered to vote before college, voting before college, and registering to vote during college and voting in college.
b) It's a correlational study--the survey method simply measured all of the variables, and a study where all the variables are measured is correlational.
c) Yes, the results support covariance because students who took a poli sci course were 9% more likely to have registered to vote in college (and 8% more likely to actually vote).
d) In my opinion, temporal precedence is not established here--students reported on whether they took a poli sci course at the same time they reported on whether they had registered to vote in college. It's not clear from this whether the course came first in time, before the voting behavior.
e) The authors did control for several likely explanations, including baseline interest in voting--and that is good. Another variable they might consider could be local political climate--perhaps some community colleges are more politically active than others. Students who attend more politically active CCs could be more likely to start to vote, and also be more likely to take a poli sci course.
f) You could recruit a sample of people and ask them if they voted in the last election. Then, you can use publicly available voting records to confirm whether or not the actually voted. You should see a strong association: People who say they voted, should have actually voted, and people who say they did not vote, should have actually not voted.