Everyday Research Methods

An experimental test of violins

Photo: Shutterstock

Stradivarius violins have an excellent reputation. Crafted in the 18th century, they are renowned for their warm, beautiful sound.But now a study has asked, are 200 year old violins from Stradivarius really superior to brand-new violins? 

You guessed it--psychological research methods can answer this question!

Before knowing how they did it, you can imagine a study that asked a group of professional violinists to play a Strad and a comparable new violin--and then to tell you which violin they prefer.

a) What kind of experimental design would this be? (Posttest only? Pretest-posttest? Repeated measures? Concurrent measures?) What is the independent variable? 

A problem with this design, of course, is that violinists' biases are likely to influence their choices. If a professional violinist already believes that Strads are better, they may find reasons to like the old violin and disregard the new. So the study by Claudia Fritz and her colleagues took precautions to keep the identity of the violins secret. Here's how the study was described by Pacific Standard:

...they decided to restrict participation to 10 world-class violinists, all award winners and experienced soloists. During two 75-minute sessions — one in a rehearsal room, the other in a 300-seat concert hall renowned for its acoustics — they played six Old Italian violins (including five by Stradivari) and six new ones.

“During both sessions, soloists wore modified welders’ goggles, which together with much-reduced ambient lighting made it impossible to identify instruments by eye,” the researchers write. In addition, the new violins were sanded down a bit to “eliminate any tactile clues to age, such as unworn corners and edges.”

In the concert hall, the violinists were given free reign: They could ask for feedback from a designated friend or colleague, and a pianist was on hand so they could play excerpts from sonatas on the various violins.

Afterwards, they rated each instrument for various qualities, including tone quality, projection, articulation/clarity, “playability,” and overall quality.

The violinists also were asked to choose only one violin as their final choice. 

b) What design did they use in this study, given this description? (Posttest only? Pretest-posttest? Repeated measures? Concurrent measures?) What's the independent variable? 

Here are the results:

Six of the soloists chose new violins as their hypothetical replacement instruments, while four chose ones made by Stradivari.

“Among these players (seven of whom regularly play Old Italian violins) and these instruments (five of which were made by Stradivari), there is an overall preference for the new,” the researchers write. “Ratings for individual quality criteria suggest that this preference is related mainly to better articulation, playability, and estimated projection (in the new instruments) — but without tradeoffs in timbre.”

c) What do you think? Do you think these results show an "overall preference for the new?" Explain your reasoning.

 

Suggested answers

a) This design is concurrent measures because they are exposed to both levels of the IV (old vs. new violin) and select only one.

b) The fact that they rated each of 10 violins after playing each one makes this a repeated measures design, with "old vs. new violin" as the IV. However one variable--selecting one of the violins--is set up as concurrent measures design. 

c) When it comes to the violin choice variable alone, the results don't seem strong enough to support the "overall preference for the new" conclusion. A new violin was chosen over an old one 6 out of 10 times. Because there were only 10 musicians in the study, it seems likely they could have gotten this result (6-4) by chance alone.  If only one of the musicians had chosen a new violin over the old, the results would have been 5-5, an even split. 

In contrast, if the study had had a larger sample size (say, 100 musicians), and 60 of them selected a new violin over an old one, that would be much more convincing. Thus, this study is an example of how sample size is an important element of statistical validity.  (Of course, finding 100 professional musicians to participate in such a study would be difficult...)

The results of the repeated measures variables (ratings of all the violins) can be seen here, in the published study (you can see the ratings in Figure 2). Do you think these results are stronger evidence in favor of the new violins, or not? 

Would you prefer a bronze or silver medal? An Olympic quasi-experiment

In this podium photo of the men's 400M Freestyle, does the smile of the silver medalist, China's Sun Yang, look happier or less happy than that of the bronze medalist, Italy's Gabriele Detti? Note: The happy gold medalist is Australia's Mack Horton.   Photo Credit: Matt Slocum AP Photo

In honor of the 2016 Rio Olympics that are happening right now, I decided to bring up a classic study on the emotional reactions of Olympic medalists. The study was covered by Scientific American a few years ago. The study showed a counterintuitive result: 

In athletic competitions there are clear winners and losers. In the Olympics, the gold medalist won the competition; the silver medalist has a slightly lower achievement, and the bronze medalist a lower achievement still. One might expect that their happiness with their performance would mirror this order, with the gold medalist being happiest, followed by the silver medalists, and then the bronze.

Psychologists Victoria Medvec and Thomas Gilovich of Cornell University, and Scott Madey of the University of Toledo think that this phenomenon can be explained by counterfactual thinking. This means that people compare their objective achievements to what “might have been.”

The most obvious counterfactual thought for the silver medalist might be to focus on almost winning gold. She would focus on the difference between coming in first place, and any other outcome. The bronze medalist, however, might focus their counterfactual thoughts downward towards fourth place. She would focus on almost not winning a medal at all. 

It is because of this incongruous comparison that the bronze medalist, who is objectively worse off, would be more pleased with herself, and happier with her achievement, than the silver medalist.

The study behind this story was a quasi-experiment. As you read the journalist's study description, decide which quasi-experimental design the researchers used: 

To scientifically investigate this question, the researchers took video footage of the 1992 summer Olympics in Barcelona, Spain. Specifically, they recorded the medal ceremonies and showed them to undergraduate students, as well as footage from the athletic competitions immediately following announcements of the winners. They asked them to rate the happiness displayed by each of the medalists on a 10-point scale, with 1 being “agony” and 10 being “ecstasy.”

On average, the silver medalists scored a 4.8, and the bronze medalists scored a 7.1 immediately following the announcement. Later in the day, at the medal ceremony, the silver medalists scored a 4.3 on the happiness scale, while the bronze medalists scored 5.7. Statistical analyses proved that both immediately after winning, as well as later at the medal ceremony, bronze medalists were visibly happier than the silver medalists.

Here are some questions about the study: 

a) What is the independent variable in this design? What is the dependent variable? (Hint: They operationalized the dependent variable in two ways).  

b) Is the independent variable a between-groups or within-groups IV? Why is the IV considered a quasi-experimental independent variable? 

c) Which quasi-experimental design appears to be conducted here?  Your choices are: non-equivalent control group posttest only, non-equivalent control group preptest-posttest, interrupted time-series, or non-equivalent control group interrupted time-series. 

d) The researchers provide enough information for you to create a graph of the results. Try it! (Which DV will you pick to graph?) Why do you think the journalist didn't provide the happiness values for the gold medalists? 

e) Quasi-experimental studies take advantage of real-world situations, but they cannot establish full experimental control.  The researchers are unable to randomly assign people to win silver or bronze medals. Therefore, what confounds might be present in this design? How might the researchers have controlled for such confounds in their study?