When the survey raises more questions than answers

(Reading time: 1m, 53s)

Yesterday, I wondered why more cultural institutions regularly evaluate the online sales path for their constituents.

I think there are a number of contributing factors — One reason is that employees don’t often have to go through the process themselves. Another is that no one person or department is charged with owning the end-to-end user experience or buyer’s journey — marketing might handle web copy, development might then be charged with future member communications, and the actual nuts and bolts interaction of buying is dictated by the software solution the museum has hired to handle transactions. And maybe no one is really aware or believes that a poor checkout experience will reduce the number of people who buy or renew online.

I was thinking about all this when I found an article by Colleen Dilenschneider called Why Expired Members Do Not Renew Their Memberships To Cultural Organizations. In a survey of lapsed members of cultural institutions, IMPACTS found that the most cited reason for not renewing is “intend to renew when next visit” — So, these former members say they were going to renew, but then didn’t because they haven’t visited again.

Chart: “Why did you not renew?”  Colleen Dilenschneider

Chart: “Why did you not renew?” Colleen Dilenschneider

It’s tempting to conclude from the results that organizations need to be sure they’re doing all they can to motivate members to continue visiting the museum — and of course visitation helps the organization meet other mission-related goals.

But what’s frustrating about the survey results is that we still don’t know why renewal is tied to visitation.

  • Is it that the members aren’t aware that they can renew online?
  • Is it too hard to renew online?
  • Have the organizations not done the best job of communicating the option to renew online ?
  • Have any of the organizations tried to incentivize renewing online?
  • Is it that some people are just giving a throwaway answer that conceals the real reason they aren’t renewing? In other words, is it easier to say, “I haven’t gotten around to visiting” than it is to say, “I really didn’t see the value in renewing” or “I’m spending that money elsewhere”?
  • Some combination of the above?

The survey results don’t answer these questions. The findings are interesting — but as soon as you start digging deeper and asking more questions, you realize how hard it is to act on the information with any confidence.

What if you interviewed some of your own members and found that they’re planning to renew during their next visit because they gave up on using the museum’s website to conduct transactions long ago?

Takeaway: Even the most thorough survey is often just a starting point. Layer in additional data sources, like interviews or user testing, to understand the particular obstacles your audience is facing.

Thanks for reading,

Kyle