A newsletter on audience research and development for cultural leaders. One reader calls it, "sometimes funny"

Last week, I suggested we question the validity (though not the intentions) of IMPACTS’ efforts to understand people’s intentions to return to cultural organizations, as shared by Colleen Dilenschneider. (You can read that first newsletter here.)

Based on some feedback I received this morning, it seems last week’s efforts were inadequate.

I know — It’s surprising. For some reason, that one newsletter did not persuade every museum leader in the world that we can’t predict how people will behave based on what they say they will do in the future.

So, let’s dig a bit deeper and examine the potential ramifications of this kind of speculative (gambling) research.

Pour me another shot of espresso, and let’s settle in for some real talk.

A waste of time for organizations that don’t have time to waste

Imagine a survey that asks people, “When do you think you’ll be able to return to your normal activities?”

Would you believe that the resulting data are indicative of when restrictions around this pandemic will be lifted?

Probably not.

You would turn to experts — scientists and health professionals — to understand when society may return to normal and, by extension, maybe when people will begin visiting your organization again.

Now, imagine issuing a survey to your own museum constituents, asking them, “When do you intend to visit us again?”

Even if they knew what they will do (they don’t), what would you really be measuring? How much are the responses really just suggestive of when people think (or hope) activity restrictions will be lifted?

It would be inadvisable to survey your own visitors in that way. The data doesn’t magically become more accurate or reliable just because you survey more people over a longer period of time.

I get that we’re all looking for some signs of hope right now, but I worry that data like this leads cultural leaders to believe that questions about future behavior are acceptable. I worry that this kind of research signals to cultural leaders that it’s ok to ask questions about what people will do months or years from now — or that you can hold a focus group and ask people what sort of membership they might buy one day and that will yield some reliable information that you can plan around.

Everything is contextual — including statistics

What’s behind all this?

Let’s forget about IMPACTS for a minute.

Why is gambling so pervasive among cultural institutions?

Is it rooted in some inferiority complex within the social sciences? A misguided but persistent view that the soft sciences are somehow less rigorous? Where does this blind faith in math come from?

Sound methodology doesn’t matter if the context doesn’t support the effort. You can construct a house perfectly — follow the blueprint exactly — and find yourself in a heap of ruins if you build that house on the beach.

How many people intend to visit next week or next month? What does it matter if they can’t safely leave home?

Imagine Disney building a theme park on the moon and then surveying people to find out how many intend to build a spaceship and fly there for a visit. They can’t do it, so why ask?

Alternatives to gambling

Okay, Kyle. Put down that eighth shot of espresso and tell us: What could we be doing instead of surveying people about future behavior?

We could be studying how people have responded to pandemics in the past. That information would likely be more suggestive of what people will do in the coming months.

Or we could be researching how organizations are responding to the current crisis to give leaders a better view of how they might respond — What their options are and how we might define success in a world without IRL visitors. Slover Linett just announced that they’re embarking upon a project of that nature.

I make these suggestions to show that there are other ways in which research firms can help cultural organizations today. I understand that IMPACTS is in the survey business — quant, preference research is their jam — so they’re not likely to pursue other avenues. But bringing this kind of speculative survey data to an industry in crisis is like bringing porn to a bar mitzvah — Is the kid going to appreciate the gift? Sure. But there are other options, and if those aren’t your thing, then maybe sit this one out.

Our work here is done

Yep. This was it. This was the newsletter that put an end to gambling once and for all.

The deafening silence you hear is the sound of museum staff everywhere putting down their cards and walking away from the slot machines. The casino has closed.

Ok, probably not.

Because I’m likely preaching the converted — If you’re reading this, you may be nodding your head in agreement. (The naysayers unsubscribed long ago.)

It’s not us I want to reach — It’s all of them.

So, spread the word. You can share these words or write down your own thoughts. (If you do, let me know so I can share your thoughts with others.) And don’t be afraid to speak up when you see people gambling at your organization.

As always, reply to this email to let me know your thoughts or leave a comment below.

Kyle

You’ve successfully subscribed to SuperHelpful Letters
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Your link has expired
Success! Check your email for a magic link to sign-in.