The Harvard Business Review described a study that tested biases among people who read internship applications. Not just any internships, either: These are internships at prestigious law firms. The HBR reports that these internships..
... open doors to even more lucrative employment in the private sector as well as prestigious judiciary and government roles. For these reasons, employment in top law firms has been called the legal profession’s 1%.
Here's a preview of the design of the study:
Now imagine four applicants, all of whom attend the same, selective second-tier law school. They all have phenomenal grade point averages, are on law review, and have identical, highly relevant work experiences. The only differences are whether they are male or female and if their extracurricular activities suggest they come from a higher-class or lower-class background. Who gets invited to interview?
a) Given this brief description, identify the design of the study. What are the independent variables? What are the levels of each IV? What is the dependent variable? Do you think the IVs were manipulated as independent groups or within groups?
Now, here's some more detail:
We uncovered this through a field experiment with the country’s largest law firms. Specifically, we used a technique — known as the resume audit method — that is widely seen as the gold standard for measuring employment discrimination. This method involves randomly assigning different items to the resumes and sending applications to real employers to see how they affect the probability of being called back for a job interview. All in all, we sent fictitious resumes to 316 offices of 147 top law firms in 14 cities, from candidates who were supposedly trying to land a summer internship position. All applicants were in the top 1% of their class and were on law review, but came from second-tier law schools.
We signaled gender by varying the applicant’s first name (James or Julia).
...to capture the economic component of class, our lower-class applicants received an award for student-athletes on financial aid. To incorporate its educational competent, they listed being a peer tutor for fellow first-generation college students. By contrast, our higher class candidate pursued traditionally upper-class hobbies and sports, such sailing, polo, and classical music, while the lower-class candidate participated in activities with lower financial barriers to entry (e.g., pick-up soccer, track and field team) and those distinctly rejected by higher-class individuals (e.g., country music). But crucially, all educational, academic, and work-related achievements were identical between our four fictitious candidates.
Now here are the results:
Even though all educational and work-related histories were the same, employers overwhelmingly favored the higher-class man. He had a callback rate more than four times of other applicants and received more invitations to interview than all other applicants in our study combined. But most strikingly, he did significantly better than the higher-class woman, whose resume was identical to his, other than the first name.
Take a look at the bar graph of the results on the HBR story site here.
Additional questions
b) How were the IVs and DVs operationalized?
c) Interrogate the construct validity of the operationalization of social class. Do you think this manipulation was accurate and valid?
d) Using the three criteria for causation, interrogate the internal validity of the study. What controls do you notice that reduce design confounds. Is there any concern about selection effects? Why or why not?
e) What interaction in the results shows a moderator?
f) Using the data provided in the story, do you think that the study showed a main effect for gender? A main effect for social class? How about an interaction? Describe each of these effects.
g) Finally, the researchers dug deeper by conducting a follow-up study to investigate why the law firms were biased. Read this section of the article and decide, What mediating theory does the follow-up study propose?
Why did the higher-class man do so much better than the higher-class woman? To further explore this issue, we conducted a follow-up experiment with a sample of 210 practicing attorneys from around the country. We asked each attorney to evaluate one of the same resumes we used in our field experiment and...rate their candidate on factors proven to influence how favorably people view job candidates but that vary between men and women. These included perceptions of the candidate’s competence, likability, fit with an organization’s culture and clientele, and career commitment.
Just like the employers in our audit study, the attorneys we surveyed favored interviewing the higher-class man above all applicants, including the higher-class woman. This time, though, we were able to understand why. Attorneys viewed higher-class candidates of either gender as being better fits with the culture and clientele of large law firms; lower-class candidates were seen as misfits and rejected. In fact, some attorneys even steered the lower-class candidates to less prestigious and lucrative sectors of legal practice, such as government and nonprofit roles, positions that tend to be more socioeconomically diverse than jobs at top law firms.
Thanks to Dr. Kathleen Lewis of Point Park University for sending this example my way and for writing the questions!
