Day 057Working with AI4 min readChinese original ↗

The more complete the prompt, sometimes the worse

Prompt 写的越完整,有时候效果更差
TL;DR
  • After turning my review method into a Skill, I found the results were actually worse than just discussing with ChatGPT: posts came out more complete and tidy, but the main point got sanded down and the voice faded. A more detailed prompt doesn’t mean better results.
  • The problem isn’t only too many rules, but no priorities. Adding a rule every time you hit a pitfall turns a one-off accident into a permanent constraint on every task.
  • I split a Skill into two layers: a fixed layer for methods that don’t change, and a run-time layer that supplies the focus on each call. When the model changes, retest the prompt; when rules pile up, reorganize it.

“A boss who controls every detail stifles employees’ creativity; micromanaging usually gives worse results.”

The last post discussed how, after a model upgrade, the same prompt may no longer work the way it used to.

This one is about something else:

the model hasn’t changed, yet we may keep editing a prompt until it gets worse.

I had turned the RedNote review method I use over and over into a Skill.

The intention was simple: since I check the main thread, factual basis, repetition, AI tone and the author’s voice every time, why not fix them in place and call them directly from now on?

But in actual use, I often felt the results from calling the Skill were worse than just discussing with ChatGPT.

Not because it didn’t follow the requirements.

Posts usually came out more complete and tidier, with every standard seemingly taken care of. But the main point got sanded down and the voice faded. What I ended up with was a post without obvious mistakes, but without much personal judgment either.

That made me realize the effect of a prompt or Skill doesn’t depend only on how detailed it is.

Roughly:

result = f(model, Skill, current input, context, tools, runtime parameters)

The model provides the basic capability.

The Skill provides a reusable set of methods, rules and resources.

The current input decides what specifically needs solving this time; the context affects how the model understands the task; and tools and runtime parameters change what it can do and how much it’s willing to put in.

So a Skill doesn’t act on its own, and it isn’t true that “the more complete it is, the more stable the result.”

A Skill can capture experience, but it can also capture the wrong priorities along with it.

It’s just one variable in the whole system.

Every standard being reasonable doesn’t mean they should all matter equally

For example, a post might be required to be all of these at once:

  • dense with information;
  • logically complete;
  • natural in tone;
  • keeping real examples;
  • not too long;
  • every judgment backed by evidence;
  • free of all repetition;
  • understandable to readers without background.

Each requirement holds on its own.

The problem is that they inherently involve trade-offs.

Keep more examples and it may get longer; explain fully enough and the pace may slow; pursue rigor and you may sometimes weaken personal expression.

When a dozen standards are all written as equally weighted “musts,” the easiest thing for the model to do is take care of each one evenly.

So every part is passable, and the part that really matters doesn’t stand out.

The problem isn’t only too many rules, but no clear priority among them.

Adding a rule every time you hit a pitfall leaves side effects too

This is very common in Claude Code as well.

The model once skipped the tests, so you add “must run tests.”

It once refactored a big chunk while it was at it, so you add “absolutely don’t modify anything outside the specified files.”

It once created lots of temporary files, so you add “don’t create any new files.”

Each rule has its history.

But one anomaly isn’t necessarily fit to be rewritten as a permanent rule.

For example, “don’t create new files” can prevent process clutter, but it may also make the model avoid the right approach when it genuinely needs to add a test file.

“Only modify the specified files” can limit scope, but it may also stop it from fixing a root cause that sits in a shared module.

The problem here is:

one specific accident got generalized into a rule for every task.

And the Skill reliably carries that rule into every call afterwards.

That’s exactly its value, and also its risk.

What belongs in a Skill?

I lean toward splitting a Skill into two layers.

The fixed layer: what doesn’t change over time

For example:

  • don’t invent facts;
  • separate facts, inferences and unknowns;
  • a particular verification must be done;
  • particular tools or materials must be used;
  • output must meet a clearly defined format;
  • a few truly stable orders of judgment.

These requirements usually still hold for a different post or a different project.

The run-time layer: what you only know this time

For example:

  • what you most want to solve this time;
  • who the target readers are;
  • what the biggest open question is right now;
  • what must be kept;
  • whether rigor, pace or personal expression matters more this time;
  • which standards should take priority.

This information shouldn’t be decided in advance by the Skill; it should be supplied each time you call it.

You can think of it as:

the Skill fixes the method; the current input decides the focus.

A good Skill isn’t the one with the most standards

What a Skill really needs to provide isn’t an ever-longer checklist, but three things.

1. A clear goal

What exactly is this Skill helping the model accomplish?

Don’t make one Skill take on reviewing, rewriting, fact-checking, title generation and style imitation all at once.

2. Priorities

When standards conflict, what matters most?

Without priorities, the model can only spread its attention evenly by itself.

3. Room to vary

Don’t permanently write a trade-off made for one task into a rule for every task.

If a Skill isn’t working well, your first reaction shouldn’t necessarily be to add more rules.

First check:

  • whether it’s taking on too many goals at once;
  • whether every requirement is written as equally important;
  • whether some rule came from a one-off failure;
  • which judgments should be supplied at call time instead;
  • whether the current model, context, tools and runtime parameters have changed.

The last post said: when the model changes, retest old prompts.

This post says: prompts also change themselves through constant patching.

Put together, the conclusion is quite simple:

when the model changes, retest the prompt; when the rules pile up, reorganize the prompt.

Written in Chinese and translated into English with the help of AI. Read the original