A friend just pointed me to this article on a Washington Post survey of local issues in my hometown of Washington, D.C. Seems there’s a little hullaballoo about the framing of the survey results—namely, that 71% of DC area residents rate the problematic Metro subway system as “excellent” or “good.” As someone who regularly does surveys for public relations (and a former Metro commuter), here are my thoughts on the matter
- The data are presented correctly. There is some grumbling that the question is asked of all residents, not those who often (or ever) ride Metro. Sure, it’s pretty useless to include the 13% of residents who never ride Metro in this denominator—but since they specify that it’s 71% of residents, not riders, it’s not “wrong.”
- The data are presented in a way to make Metro look good. Of course they are. Who’s going to be impressed if they read that only 15% of DC area residents, whether they ride Metro or not, think it’s “excellent?” This piece combats the backlash with the point that “76 percent of D.C. area riders who ride Metro ‘very often’ still rated it positively.” If they mentioned that this “76%” is actually only 118 people (76% of the 14% or 155 who ride very often, this doesn’t look as good. The folks in Metro’s PR department want to paint the system in the best possible light. Can you blame them?
- If Metro wants to use the data to improve their service instead of just for public relations, they can. I was actually delighted to see that this survey asked for reasons why people don’t use Metro. We probably won’t see a headline that 44% of non-riders don’t ride because Metro is too expensive (although props to the Washington Post for publishing these results). But, Metro now has this data at their fingertips and can use it to make changes.
In sum: there are huge differences between using data for public relations and collecting data that can be used to actually improve a service. PR surveys should be taken with a grain of salt– and the burden is on the reader to dig deeper. Surveys that include questions on specific problems or potential improvements provide opportunities to get better. Metro’s survey seems to have done both.