Roger Craver: Which metrics matter (& don’t)

I asked a dumb-ish question about metrics. Super-guru Roger C. answered … wisely, kindly AND in SHOCKING! depth. I learned a ton. Save this one for the files.

My emailed question to Roger, co-editor of The Agitator, asked: Is “cost per dollar raised” finally extinct as a rusty/rustic metric? I get to preach to thousands of fundraisers annually. Right now I tell them that “Lifetime Value [LTV] is gospel, LTV is the CORE metric.” But I don’t really know. Is it? Is LTV the metric of choice?

“Roger Craver here….”

[he wrote] Good and extremely important questions you ask.

I’m so glad you’re looking to do some preaching on the subject … because there’s a lot of apostasy, heresy, and plain theological ignorance out there.

Cost per dollar raised

“Cost of dollar raised” is what I term a “vanity” metric, [Roger said.] Meaning a measure that’s largely useless … because asking and answering the question isn’t of any significant help in steering toward a successful long-term future.

The cost for raising a $1 in a year-end appeal to the best donors will be quite different than the cost of raising a $1 in acquisition or through a sustainer invitation.

I have never understood why boards even bother to ask the question other than they have some misguided hang-up about efficiency, fundraising overhead or some equally irrelevant fear.

After all…

  • The cost to raise a single $ for a local food bank through a neighborhood bake sale may be 7 cents and raise enough money to feed 8 people for three days.
  • While the cost for Food For the Poor may be 20 cents and feed 2 million people.

Key Performance Indicators (KPIs)

KPIs are too often used at a crutch.

Most organizations have far too many of them. (True story. Spotted on a men’s room wall: If it’s easy to measure and count then “metricize” it.)

In my [i.e., in Roger’s] view, KPIs exist to reduce the complexity of organizational performance.

Generally, I use three KPIs;

  • one for long-term financial steering (Lifetime Value [LTV]);
  • one for operational measuring (campaign return on investment [ROI]);
  • and one for health/financial growth of the donor base (Commitment or Loyalty).

Commitment/Loyalty. This is the most predictive of the metrics.

It measures donor attitude. And it’s attitude that affects the donor’s behavior in the future.

The problem with this metric is that few organizations will take the time and expense to measure and score it. Those who DO have seen enormous benefits in revenue, retention and cost savings. (You can see more on “Commitment” in my Retention Fundraising book.)

To answer your basic question….

[Roger generously continued…]

If there’s one metric I’d dwell on and teach others about it’s Lifetime Value (LTV). Since you have me wound up on the subject, here’s a bit more thought on metrics….

What we MEASURE vs. what we MANAGE

It’s fashionable these days for many nonprofit fundraisers and their consultants to claim they are “data-driven.”

The problem is [40 years of front-line experience talking here]: MOST focus on the “data” part of that slogan.

Few understand [he said] the requirements of the second word – – “driven.”

In reality, what we choose to measure will likely reflect on what we choose to do or choose to manage.

THE BOTTOM-LINE DIVIDE: Vanity vs. Value

It’s very important that we choose metrics that really matter in terms of being helpful — or even fundamental — to growing our donor database and our income.

I divide data between what I call “vanity metrics” and “value metrics.” The distinction between these types of metrics lies in the question:

“What would I do differently based on the information provided by that metric?”

If that question doesn’t arise around the data you’re collecting, you probably shouldn’t be spending time or money to collect and analyse them.

In essence, part of the problem with “vanity metrics” is that nobody does anything with them.

More importantly, and dangerously, “vanity metrics” are often used to drive absolutely bad decisions. For example: “If I spend more on online search advertising, can I drive up the number of website visitors?”

A largely meaningless strategy. A largely meaningless result!

Some great examples of vanity metrics

Benchmarks. My complaint with benchmarking is that few organizations use it as a questioning metric to guide or develop new actions. I applaud use of benchmarks and the organizations that do use it for that purpose.

Unfortunately, too many fundraisers use benchmark as a proud justification for doing nothing except boasting to their board or CEO. As in “we’re doing great! We’re above average in the benchmark for retention.”

If ever a worthy metric concept was misapplied for the sake of vanity, it’s the benchmark. In the social media age, here are some vanity metrics I’m sure you’ve heard bandied about day after day ad nauseum:

Number of hits. A fossil from the early days of the web. If you have a big site with many elements, there will be lots and lots of hits. So what? Better to count the actual number of people involved. I.e., number of page views.

Number of visits or number of unique visitors. Sheds no light on why they visited, why they stick around, why they left. Pretty much worthless.

Number of likes, friends, followers. Simply a popularity contest. Only if you know how many followers will take action when you make a request do you really have an actual metric. Most organizations don’t use it this way.

Another of my pet vanity metrics, which will surprise some, is acquisition response rate.

An amazing number of direct response fundraisers measure “success” or “failure” by response rates as in “our rival’s acquisition gets only a .65% rate and we get 1.2%!”

Pretty much worthless because it tells you only how a specific campaign went, and raises almost no “what should I do differently” questions, other than “what should I do differently to get the response rate up?”

[“GUILTY!” I murmured into my upturned winter collar. A cold day getting colder.]

What all this reflects is a singular obsession [Go, Roger!] with nothing but campaign metrics and various manipulations that produce only variations on the same numbers. Columns and columns of redundant, non-useful metrics – – created by just switching around the numerator and denominator of response rate, average gift, average cost AND…that magic output: cost to raise a dollar.

1 Comment

  1. Dafny February 1, 2018 at 1:00 pm #

    Tom,
    Thank you for posting this. I am all too familiar with the obsession on KPI’s especially as someone who went from digital ad operations to fundraising.
    I always feel like I’m chasing after that magic industry standard response rate even when a project gets fully funded fast.
    I certainly appreciate you sharing Mr. Craver’s perspective on how we measure success.

    Reply

Leave a Reply