Who’s afraid of key performance indicators?

(Reading time: 2 minutes, 52 seconds)

A newsletter by Bob Lalasz keeps floating to the top of my inbox. Bob is the founder of Science+Story and a fellow member of The Expertise Incubator, which I joined earlier this month.

The subject of the email is “If You’re Afraid of KPIs,” and you can read it on the Science+Story blog. Here’s one part that had me nodding along:

I’ve seen organizational leaderships doing everything they can not to set key performance indicators for their communications.



I’ve seen leaderships setting absurdly easy-to-achieve KPIs.



And I’ve seen leaderships (and funding officers) who treat missing a KPI as occasion for a Zero Dark Thirty interrogation.



All of these scenarios flow from the same places. Fear of failure; fear of punishment for failure.

In yesterday’s letter, I described how museums can default to tracking simple metrics like website visits or pageviews. Museum folks may be justifiably frustrated if a funder uses a simple metric like bodies-through-the-door as a heuristic for value. However, those same decision-makers may use similar metrics internally — like pageviews and visits — to measure the success of their communications.

You can place every kind of measurement into two buckets — production and consumption. Organizations can measure what employees produce or they can measure how their work is consumed. (A kind of ugly word, but stick with me.)

In an environment where success is measured in terms of production, it’s all about output — more exhibitions, more events, more newsletters, more social media posts = good. It’s a formula that’s easy to use and understand. Things get scary when you start measuring consumption. Is anyone viewing/reading/watching this stuff? How? What can we surmise from that behavior?

And, as weak as bodies-through-the-door and eyeballs-on-the-content are as a way of measuring consumption, they do fall into that consumption category. It’s arguably a step above production-focused environments.

But the challenge is to go beyond gross units of measure and look at what happens once they’re through the door or on the website — and to define who “they” are. How do people who arrive on a school bus behave differently than people who took a cab? How do people who came from a search engine behave differently than those who came from a newsletter? How does that activity compare to an ideal outcome?

And I think that’s where the fear Bob talks about can come into play. You don’t have to be in a fear-driven environment to feel intimidated at the thought of defining ideal (behavioral) outcomes.

In response to yesterday’s video, a list member (who I neglected to ask if I could quote, so they’ll remain anonymous here) wrote in:

What you propose will still generate numbers and you will never know why people behaved in the way they did, and you will never know how the website (or exhibition) affected them. Thoughts??

That’s true. Studying behavior from a distance can be incomplete, but behavior can still act as a performance indicator. A hurried visit to through a gallery or a bounce on a web page can indicate a weak interaction; A full day in the museum or a purchased membership does reflect a desired outcome.

But it’s still a good point. To really understand motives and qualitative outcomes — to get a complete picture of the value of the organization’s efforts to an individual — you need to go beyond behavioral data to get to qualitative insights. You really have to talk to people, and my email yesterday didn’t address that.

In an ideal world, all these methods of research — from behavioral research to audience research — would be continuously informing one another to help the organization understand performance, but I think change has to be incremental. It doesn’t take a fear-driven culture to suffocate experimentation and growth. Being buried up to our eyeballs in work and feeling like we’re always a little behind will do just fine.

Thanks for reading,

Kyle

PS. Here’s how you can join Bob Lalasz’s mailing list. 👍🏻👍🏻

Kyle Bowen