Day 077Product & Design5 min readChinese original ↗

When a measure becomes a target: Goodhart's Law

当指标变成目标,它可能就失效了:我最近重新理解 Goodhart’s Law
TL;DR
  • A metric is meant to help me see the goal, but once I treat it as the goal — “this post has to get so many likes,” “apply to 5 jobs a week” — my behavior follows the number, and may end up further from what I actually want.
  • This is especially true in product work: a proxy is often clearer and easier to act on than the real goal, so as you go, it’s easy to end up optimizing only the things that are easiest to measure.
  • Now, when I come across a metric I care about, I ask one more question: if I make this number really good, will the thing I actually want to improve necessarily get better? If not, it may just be the finger pointing at the moon.

Recently I came across a concept again: Goodhart’s Law.

The line people most often use to sum it up now:

When a measure becomes a target, it ceases to be a good measure.

When a measure becomes a target, it often stops being a good measure.

It struck a chord when I saw it, because I’d been thinking about a similar question for a while.

When subjective feelings and quantifiable metrics conflict, which should you trust?

Now I think metrics still need watching, of course; it’s just best to know first:

what exactly it’s helping me measure.

01 Metrics were only ever meant to help us see the goal

Take writing content.

What I really want is to keep writing things I think are valuable, that others can get something from after reading.

Reads, likes, saves and new followers all matter, because they tell me:

  • what people are willing to read;
  • what resonated;
  • what I may not have explained clearly.

But if it slowly turns into:

“this post has to get so many likes,”

behavior easily changes with it.

Titles start drifting toward whatever gets clicked, you only write topics the data has already validated, and a piece you really like but whose numbers are ordinary may get judged “not worth writing.”

The numbers aren’t wrong.

It’s just that they were meant to help me observe results, and later started taking part in deciding what I do.

Job hunting is the same.

Say I set myself an execution rule:

apply to 5 jobs a week.

It’s very useful; it stops me from endlessly researching and preparing and never actually acting.

But “finding work that suits me” and “completing 5 applications a week” are clearly not the same thing.

If I start chasing the second number, I could easily apply to a few jobs that don’t fit so well and hit the KPI nicely.

The number got better, but the problem I really wanted to solve isn’t necessarily closer to an answer.

02 In product work, this is especially common

When building products, we’re very used to watching metrics:

  • DAU
  • conversion rate
  • retention
  • revenue
  • feature usage

All of these matter.

It’s just that what users really need has never been a prettier dashboard.

What we ultimately want to know is still:

  • did the product solve the problem?
  • why do users stay?
  • is the business healthy?
  • did what we built actually create value?

But these things are often hard to measure directly.

That’s why we need proxies, turning complex reality into numbers we can observe.

And that’s exactly where the trouble lies:

a proxy is often clearer and easier to act on than the real goal.

So as you go, it’s easy to start seriously optimizing the things that are easiest to measure.

03 I love the image of “the finger pointing at the moon”

I’ve long loved an image Bruce Lee used:

the finger is for pointing at the moon.

If you keep staring at the finger, you’ll miss the moon.

Thinking about Goodhart’s Law recently, it came back to me.

Metrics are sometimes a lot like that finger.

Their value lies in helping us find and observe something more important.

But when all your attention lands on the finger, it’s easy to forget what you were looking at in the first place.

There’s an even more troublesome case:

once we start frantically optimizing the finger, it may no longer be pointing properly at the moon.

04 In life, I increasingly want to first confirm what the “moon” is

Strictly speaking, what follows isn’t entirely Goodhart’s Law anymore.

But I think it’s a very similar kind of problem underneath:

could something meant to serve life slowly end up deciding life itself for us?

I once talked about this with a friend:

when choosing a city to live in, do you go to a city because of a job, or first choose the city you want to live in and then find work there?

I’ve mostly done the former.

Wherever there was a suitable job, that’s where I went.

Work largely decided for me where I lived, what I saw every day, who was around me, and what life looked like.

My friend is closer to the latter.

First think about where you’d like to live, then work out how to handle work.

I don’t now think the latter is necessarily more enlightened.

If career opportunity is what matters most in a given stage, following the work is perfectly reasonable.

It’s just that the question made me start looking at two things separately:

  • what kind of career do I want?
  • what kind of life do I want?

They’re related, but they’re not the same question.

If work was meant to support life but ended up deciding life entirely for me, I may need to recheck the order.

Buying a home is the same.

If you’re buying a home you’ll live in long term, what exactly are you optimizing for?

Investment value, rent-to-price ratio and room for appreciation matter, of course.

But some things hardly show up in Excel:

  • whether I feel comfortable coming back here every day;
  • whether I like the surroundings;
  • whether the commute and daily life are convenient;
  • whether I’d really be willing to live here for many years.

If the goal of buying a home was to find a place I’d love living in, and in the end every decision got taken over by investment metrics, that also seems a bit like studying the finger and forgetting the moon you wanted to see.

Relationships too.

Two people having friction or arguments ≠ the relationship being unhealthy.

How many conflicts happen is just a phenomenon.

What may be more worth looking at is whether the two can understand each other, solve problems and make repairs, and how each is actually doing over the long run together.

Some truly important things are hard to sum up in one number or one phenomenon to begin with.

05 So now I ask one more question

When I come across a metric I care about a lot, I now ask:

if I optimize this number really well, will the thing I actually want to improve necessarily get better too?

For example:

  • lots of applications → necessarily closer to the right job?
  • lots of likes → necessarily closer to the writing I want to do?
  • lots of features built → users’ problems necessarily better solved?
  • a home with great investment numbers → necessarily the place I really want to live?

If the answer is “not necessarily,” I won’t stop looking at the metric because of it.

I’ll just remind myself:

it may only be the finger.

What really needs reconfirming often is where the moon actually is.

Written in Chinese and translated into English with the help of AI. Read the original