In fall 2014, IU conducted its first-ever sexual assault climate survey. A year later, IU published the results, and President Michael McRobbie described how the survey’s “sobering” findings would be used to guide the University’s fight against sexual violence going forward.
Unfortunately, IU is trusting findings that aren’t completely dependable.
As stated in the final 53-page report produced by the Division of Student Affairs, all 42,085 who were students enrolled at IU-Bloomington in November 2014 were sent emails asking them to participate in the questionnaire. The questions were meant to discover what percentage of the student population experiences nonconsensual sexual acts, where those assaults occur and much more.
The red flag that seized my attention was the fact the University asked the entire undergraduate and graduate population to respond, even though surveying large numbers of people isn’t the key to reaching accurate results.
Famously, the once-popular magazine Literary Digest polled millions of Americans and predicted that Alf Landon would crush Franklin D. Roosevelt in the 1936 presidential election. As we know, though, it was Roosevelt who earned the landslide win.
The most well-known issue with the Literary Digest’s poll was that the survey ballots were sent to a sample of people that wasn’t quite representative of the voting population. Essentially, the sample was somewhat biased toward the wealthy and thus toward Republicans.
However, despite this flaw in representation, later research by Gallup indicated that a slim majority of voters who had received the magazine’s ballot were still Roosevelt supporters.
In other words, if everyone who received a questionnaire had filled it out, Literary Digest would have correctly predicted Roosevelt’s victory.
This shows that there was a second reason why the poll reached the wrong conclusion. Roosevelt voters felt less strongly about the election and often declined to return the questionnaire. As Maurice C. Bryson stated in a 1976 article in the American Statistician, “The respondents represent[ed] only that subset of the population with a relatively intense interest in the subject at hand.”
Similar to the intensity of that election, another discussion that can summon even stronger sentiments is the topic of sexual assault.
Thousands voluntarily responded to IU’s sexual assault survey, but they accounted for only about 16-20 percent of those who were emailed. I don’t know whether victims of nonconsensual sexual acts were more or less likely to respond, but it might have had an effect on who decided to participate
Either way, this attempt to survey the student body just doesn’t seem valid enough for the findings to be used by themselves “to shape future programming and strategies related to sexual assault prevention” on campus.
IU pursued tens of thousands of responses without considering whether the responses it got would be representative — not unlike how the Literary Digest sought millions of responses and made the wrong prediction.
When Bryson observed the shortcomings of the Digest poll years later, he stressed that the voluntary nature of mailed surveys makes them arguably the “worst” way to collect accurate data. Adding an “e” before “mailed” doesn’t change how right Bryson was, so IU must be careful not to take action solely based on survey results that might not accurately reflect reality.
