Annie Hall and the Dangers of Vague Quantifiers

By jessicabroome on July 22, 2013 in Musings, Research Tips
1
0

I love this scene from Annie Hall, where Annie and Alvin are telling their respective therapists how often they have sex.

Annie Hall and the Dangers of Vague Quantifiers

Alvin: Hardly ever. Maybe three times a week.

Annie: Constantly. I’d say three times a week.

I always encourage clients to watch this clip when they want to ask questions about frequency.  Too often, people want to use vague quantifiers like often, sometimes, occasionally, and rarely in survey questions.  Here’s an example:

 

In the past month, how often did you get at least 30 minutes of exercise?

Regularly * Sometimes * Occasionally * Never

As a self-proclaimed “gym rat,” I consider 5-6 times a week “regular” exercise; anything less than that is “sometimes.” Other people consider 3 times a week “regularly.”  On the other hand, if you asked me this question, I would interpret it very differently:

 

How frequently do you clean your home?

Regularly * Sometimes * Occasionally * Never

I’m not sure if this question is asking me about the past week or the past year, but on average, I clean once a week—that’s regular enough for me.  Plenty of people would call that “occasionally,” and insist that “regular” housecleaning is done every day. In short: these terms can be interpreted wildly differently by different people– or even by the same person in different contexts.

 

Questions with vague quantifiers are easy to write and don’t require a lot of thought. But you rarely have any idea how your respondents are interpreting these words or what frequency they think they are conveying.

There are situations where this type of question can work, but if you really want to gauge frequency, I’d offer two things that you need to think about:

What is an appropriate reference period?  Are you interested in knowing about behavior in the last year? The last week? The last 24 hours?

What ranges would be reasonable answers? Do your response options start with “fewer than 3 times” or “fewer than 20 times?” Is the high range “more than four times” or “more than 100 times?” Or is this something people might do “more often than once a week” to “less than once a year?”

At the risk of sounding like a broken record, I’d recommend pre-testing to make sure you’re getting this right!

About the Author

jessicabroomeView all posts by jessicabroome

1 Comment

  1. Stanley August 5, 2014 Reply

    Appreciating thhe persistence you put into your blog and inn depth information you present.
    It’s nice to come across a blog every once in a while that isn’t the same oldd
    rehaswhed information. Excellent read! I’ve bookmrked your site and I’m addkng your RSS
    feeds to my Google account.

Add comment

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

All material copyright 2014-2016 | Jessica Broome Research | Portrait photo by Sarah Hodzic, Blink Photography