Everyday Research Methods

What if you meet your spouse online?

There's lots to work with in this Washington Post story, which reports on a study that polled married people about how they met their spouses.

According to the Post, the study contacted about 200,000 people via email, and about 20,000 of them responded to complete the online survey. There seem to be two main results of this survey.

First, the story reports that

More than a third of the selected respondents reported meeting their spouse online. About half of that third met through online dating; the rest met through other venues such as chat rooms, games and virtual worlds.

a) Can you use this result to support the frequency claim that about 1/3 of all married Americans have met online?  Why or why not?  What validities will be most important as you consider this frequency claim? What information do you  need to know?

 

The second major result from this study appears to be the following:

The online marriages were durable. In fact, people who met online were slightly less likely to divorce and scored slightly higher on marital satisfaction.

b) What kind of claim is this conclusion making ? What are the variables in this claim?

c) Sketch a bar graph of the result that is being described.

d) Apparently, the authors of the study wondered if demographic factors such as income or ethnicity might have explained why online marriages were higher in satisfaction. According to the following paragraph, did they rule out such alternative explanations?

After controlling for demographic differences between the online and real-world daters, those differences remained statistically significant...

e) The journalist obtained a quote from another psychologist (Dr. Harry Reis), who also reacted to this association. The psychologist was quoted,

“They did control for demographic factors,” he said. “But they did not control for personality, mental health status, drug and alcohol use, history of domestic violence, and motivation to form a relationship.” All of these factors are known to affect marital outcomes, he said.

How could the authors of the study rule out factors such as "motivation to form a relationship" in a future study?

f) Because this study was covered widely in the press, it provides a chance to see how a variety of journalists covered the story. Which of the following headines seems most appropriate, given the design and results of the study?

Meeting online leads to happier and more enduring marriages

Online dating leads to "more successful" marriages, study

More satisfaction, less divorce for people who meet spouses online

Marriages among those who met online are as stable as others, study says

Suggested answers

a) For a frequency claim, you need to interrogate construct validity and external validity. First, you'd want to ask about how accurately the scientists measured whether pepole had met their spouse online. Second, you'd need to know if the sample of 20,000 is representative. This is a very large sample--much larger than a typical, randomly sampled telephone poll would use. So it's important to ask whether these 20,000 sampled respondents represent all married Americans before you can generalize the results to that larger population. Fortunately, you can read the published journal article online and find out the details of the sampling.

b) This is an association claim--the two variables are "whether or not you met your spouse online" and "level of marital satisfaction."

c) Your sketched bar graph would have categories for "how people met their spouse" on the x axis and "Marital satisfaction" on the y-axis.

d) Apparently, they used multiple regression to rule out those demographic factors as third variables.

e) Dr. Reis's idea could be tested by measuring some of these personality variables in a large sample, as well as asking people how happy their marriages are and whether or not they met online. Regression analyses could control for the personality variables and see if there is still a relationship between meeting online and having a more happy marriage.

f) Of these, the third headline seems best. For one, it captures the correlational nature of the study (this headline makes an association claim, while the first two make a causal claim). It also gets the results right. The fourth headline is also an association claim, but in fact the study found that online marriages were a little more happy, so the fourth headline doesn't depict the results accurately.

How Do Young Parents Feel about New Media?

This recent piece in USNews describes the results of a poll about how young parents feel about their kids' use of smartphones and tablets.

Let's use this story to practice what you are learning about surveys and polls.

Here's one excerpt from the journalist's story.

Surveying more than 2,300 parents of children up to age 8, researchers from Northwestern University found that the vast majority -- 78 percent -- report that their children's media use is not a source of family conflict, and 59 percent said they aren't concerned their kids will become addicted to new media.

a. Will this sample of 2300 parents be able to generalize to American parents overall?  What do you need to know?  (and, what kind of validity are we asking about here?)

b. Based on the report above, what kind of question do you think the researchers asked--a forced choice question? An open ended one?

Here's another excerpt:

"We asked parents what their challenges were as the parents of young children . . . and sometimes media was never mentioned," said study author Ellen Wartella, director of Northwestern's Center on Media and Human Development. "Parents of children this age are concerned about their health, safety, nutrition and exercise, and media concerns are much lower down the list. That was a surprise."

c. Based on this information, do you think the "challenges" question was closed ended? open ended? Do you think they used a construct valid way to ask this kind of question?

 

Finally, here's one more part of the story:

The notion that parents are apt to shush their kids by handing them a smartphone or tablet also appears to be false, according to results. To keep their children quietly occupied, moms and dads said they were more apt to turn to toys or activities (88 percent), books (79 percent) or TV (78 percent). Of parents with smartphones or iPads, only 37 percent reported being somewhat or very likely to turn to those devices.

A reader of this news article submitted the following comment:

Parents never tell the truth in situations like this. Either they hate to admit that they shut little junior up by putting him in front of a phone, or they don't even realize how much they do it.
Look around you in restaurants and just watch how many parents with small children let them sit there playing with mom's phone or a Nintendo DS.

d. Which validity is the reader criticizing?  What do you think of the reader's comment?

Suggested answers:

a. 2300 is probably plenty--as long as it is a sample that was obtained via some probability sampling method (such as random sampling or cluster sampling), 2300 is an adequate sample size. Remember--when it comes to external validity, it's not how many are in your sample, it's how you got it.

b.We can't be sure without checking, but I suspect this was a closed ended question, something like, "is your child's media use a source of family conflict?" or "to what extent is your child's media use a source of family conflict?" 

c. In this case, it seems clear that the question was open-ended, such as, "what are your challenges as a parent?"or, "Tell us about some challenges you face as a parent. If you are interested in measuring what parents think about first when it comes to parenting challengies, an open-ended question might be the best. If you asked them, "is media use your biggest challenge?" you might get more parents to say "yes" than if you simply ask them, "what are your biggest challenges?"

d. This reader is criticing the study's construct validity. While it is probably very easy and efficient to ask parents to self-report their ways of calming their kids, the reader has a good point, in my opinion. Parents probably do not want to admit that they quiet their children with a smartphone or tablet. A better way to find out what parents do to keep kids quiet is by visiting a few restaurants and observing what children are doing. Can you design a study that would do this? How can you be sure your study has good external and construct validity?