Words And What They Mean

By jessicabroome on January 1, 2013 in Quantitative Research
0
0

Survey design is a linguistically intense undertaking.  Every respondent should get the same meaning from your questions—and it should be the meaning you had in mind when you wrote the questions. A client wanted to know about feelings of security in different neighborhoods, and wrote the question “How safe do you feel in the area where you live?”  In pre-tests, we quickly learned that an area is not an area is not an area! Some respondent quotes:

 

Words And What They Mean“I don’t feel at all safe in this area of the country!  We get hurricanes all the time.”

“I feel extremely safe in my building and on my block, but not on the next block over.”

“I feel somewhat safe in this town.  There is some crime, but not really in my neighborhood.”

 

So what can you do to standardize question meaning? A few things:

  1. Pretest! Without pretests, my client would have plowed forward with “area—“ and respondents would have understood it in all different ways. Ideally, pre-tests are conducted by a trained interviewer with a number of respondents to test for consistency of understanding.
  2. Expert interviews: Your expert could be your mom, a dentist, or your fifth-grade neighbor—as long as they’re typical of the people you want to take your survey.  Do they understand the questions? Are these the words they would use to talk about these ideas? Even a quick conversation could shed some light on potential problems with your wording.
  3. Test the grade level of your questions with a tool like the Flesch Reading Ease test: http://www.readabilityformulas.com/flesch-reading-ease-readability-formula.php (keep in mind that the average reading level in the US is around 8th grade, and one in five Americans reads at or below a 5th grade level).

About the Author

jessicabroomeView all posts by jessicabroome

0 Comments

Add comment

Leave a Reply

Your email address will not be published. Required fields are marked *

*

All material copyright 2014-2016 | Jessica Broome Research | Portrait photo by Sarah Hodzic, Blink Photography