Step one in learning AI: don't ask which model is best

First, separate the two: what’s model capability, and what’s agent capability?

It sounds a bit technical, but it’s actually very basic.

We say things like:

• This AI is so strong.
• That AI is so dumb.
• This model can write code.
• That tool really gets me.
• This agent can work on its own.

But if you take it apart carefully, a lot of the time the “strength” we feel doesn’t come entirely from the model itself.

• It may come from the model.
• It may also come from the memory system.
• It may also come from context management.
• It may also come from tool use.
• It may also come from the product’s ability to break a task down, execute it, check it and roll it back.

So these days I’d rather ask a more specific question:

when we say an AI is strong, which layer is it actually strong in?

My understanding:

model capability is the AI’s ability to “think” within the current context.

Agent capability is the product system’s ability to let the model “get material, use tools, take actions and keep moving forward.”

The two often show up layered together, so they’re easily confused.

But they aren’t the same thing.

A crude but effective test:

If you take away all the history, files, tools and memory, and give the model only the current text, can it still do the task?

If it can, it’s mostly model capability. If it can’t, it most likely depends on agent / product-system capability.

What is model capability?

Model capability is more like “the brain’s basic abilities.”

For example:

• understanding language;
• reasoning;
• writing;
• summarizing;
• translating;
• generating code;
• recognizing relationships between pieces of information;
• making judgments based on the material at hand.

You give AI a passage in Chinese and ask it to turn it into English. That’s model capability.

You give it an article and ask it to summarize the argument. That’s model capability.

You ask it about a concept, like what RAG is, what an agent is, what distillation is. That’s mainly model capability too.

Because it doesn’t need to know what you said before, nor open any outside tool. As long as the current context is enough, it can do it.

So a strong model means that “once it sees the material,” it understands, reasons and generates better.

The problem is, real tasks are rarely that clean.

A real task usually isn’t:

“Please answer an isolated question.”

It’s:

“Based on what I’ve said before, and where this project stands now, keep pushing it forward.”

At that point, model capability isn’t enough.

What is agent capability?

Agent capability is more like the working system around the model.

It determines whether the model can:

• find past information;
• read the current files;
• call tools;
• search the web;
• run code;
• access a calendar or email;
• modify files;
• keep executing a multi-step task;
• correct itself after failing;
• check before and after acting.

So an agent isn’t “a model that talks better.”

An agent is more like:

model + context + memory + tools + permissions + workflow + checks

Looking at the model alone, it’s like a smart person.

But only with an agent system does it start to look like someone who can get work done.

That’s a big gap.

A smart person sitting in an empty room can of course answer questions. But if you want them to finish a job, they need materials, a computer, permissions, tools, notes, checklists, and to know where the project left off.

AI is the same.

An example

You ask AI:

“Help me make this paragraph sound more natural.”

That’s model capability.

Because you’ve already given it the material.

But if you ask:

“Keep helping me with that AI memory system topic from last time.”

that isn’t pure model capability.

Because the model itself doesn’t know:

• which time “last time” was;
• what we talked about;
• which angle you prefer;
• how far the first post got;
• what the second post should pick up;
• which kind of AI tone you dislike.

At that point, the system first has to do one thing:

find the relevant context and put it into this conversation.

If it finds it well, you’ll feel:

“It gets me.”

If it doesn’t, you’ll feel:

“Why does it act like an amnesiac contractor again?”

What really shapes the experience here isn’t only the model. It’s memory recall, how context is organized, and product-level design.

Now look at coding agents

Coding agents show the difference even more clearly.

Say you ask AI:

“Fix this bug for me.”

If you just paste an error message to the model, it can guess. But however clever the guess, it’s still a guess.

What a real coding agent has to do:

• read the repo;
• find the relevant files;
• understand the project structure;
• change the code;
• run the tests;
• keep fixing based on the errors;
• confirm nothing else broke;
• finally give you a diff.

None of these steps is only model capability.

Model capability is:

• whether it can understand the code;
• whether it can work out the cause of the error;
• whether it can write a reasonable fix.

Agent capability is:

• whether it can get into the project;
• whether it can search files;
• whether it can run commands;
• whether it can remember the project’s rules;
• whether it can make the changes;
• whether it can verify the results;
• whether it knows when to stop and ask you.

So whether a coding agent is good isn’t only about whether the underlying model is smart.

It’s also about whether it has a good workplace.

• Is there a project description like AGENTS.md / CLAUDE.md?
• Are there permission boundaries?
• Are there tests and rollbacks?
• Is there historical state?
• Is there a clear definition of done?

Otherwise it’s a very smart but very dangerous intern.

Enthusiastic. Fast. But it might rebuild your whole project the way it likes.

Thanks, but no thanks.

So why does the same model feel completely different in different products?

Because the model is only the base.

The real product experience comes from a whole system.

I break it into six layers:

Layer one: Model

The model’s own abilities in language, reasoning, writing and code.

This is the foundation. If the model itself is too weak, however good the surrounding system, it’s just handing it better material.

Layer two: Context

How much it can read at once, and whether it can catch what matters.

A big context window isn’t necessarily good. Stuff in too much irrelevant information and the model gets lost too.

So the key isn’t “put in more,” but “put in the right things.”

Layer three: Memory

Whether it can keep important information across conversations and projects.

But more memory isn’t always better. Remembering wrongly, remembering outdated things, mixing things up — all bad.

Layer four: Tools

Whether it can search, read files, run code, call APIs and access outside systems.

An AI without tools can only talk. An AI with tools starts being able to do.

Layer five: Workflow

Whether it can break a task into steps, keep moving forward, and correct itself after failing.

Many AI demos look impressive because the task is short. Once a task spans a dozen steps, workflow capability becomes critical.

Layer six: Control

Whether users can see what material it used, what actions it took, what information it remembered. Whether they can undo, limit, confirm and delete.

This layer is easy to overlook. But in real products, it decides whether users dare hand things over to AI.

So “is this AI strong” can’t be judged by model leaderboards alone

Model leaderboards are useful, of course.

But if you’re an everyday user, or someone building products, looking only at leaderboards will mislead you.

Because what you actually use isn’t a bare model.

You use a system the product has wrapped around it.

It might have:

• memory;
• project spaces;
• file uploads;
• web search;
• code execution;
• tool calls;
• permission management;
• automated flows;
• review mechanisms.

All of this affects the “intelligence” you end up feeling.

So sometimes when you think an AI is no good, it isn’t necessarily that the model is bad.

It might be that:

• you didn’t give it enough context;
• it didn’t get the right material;
• it didn’t have tool permissions;
• it had no project memory;
• it recalled the wrong history;
• it had no checking step;
• the product just isn’t suited to this task.

The reverse is also true.

Sometimes when you think an AI is very strong, it isn’t necessarily that the model suddenly saw the light.

It may just be that the agent system behind it is better designed.

• It’s better at finding material.
• Better at organizing context.
• Better at calling tools.
• Better at keeping things going.
• It stops and asks you at the right moments.

That’s product capability.

What does this mean for everyday users?

At the very least, this distinction can make us a little less superstitious.

From now on, don’t just ask:

“Which AI is the strongest?”

Ask instead:

what kind of capability does this task actually need?

If you just need rewriting, translation or explaining concepts, model capability matters a lot.

If you need analysis across lots of material, context and file retrieval matter a lot.

If you need to push a project forward over time, memory and project management matter a lot.

If you need to write code, run tests and change files, tools and permissions matter a lot.

If you want AI to handle a real workflow for you, you also need to care whether it has checks, undo and boundary control.

That’s more practical than “which model is smartest.”

Because the tasks in real life don’t all need the same kind of AI.

• Some tasks need smarts.
• Some need memory.
• Some need to look things up.
• Some need to execute.
• For some, the most important thing isn’t capability but not running wild.

What does this mean for indie developers?

If you’re building an AI product, the distinction matters even more.

Many AI products start by thinking:

Which model should I use? GPT, Claude, Gemini, or an open-source model? Should I switch to a stronger API?

Those matter, of course.

But what really decides the product experience is often the layer outside the model.

What you need to design is:

• what the user inputs;
• what context the system adds;
• which history should be recalled;
• which information must not be brought in;
• when to call tools;
• what to do when a call fails;
• how output is checked;
• how users make changes;
• how state is saved;
• how the task continues;
• when dangerous actions need confirmation.

That’s the difference between an AI product and a prompt wrapper.

A prompt wrapper is:

user input → model output

A real AI product is more like:

user input → understand the task → add context → retrieve material → call tools → generate results → check → let the user confirm → update state

Everything in between is the product’s value.

And it’s where indie developers can really make a difference.

Because it’s hard to train a stronger base model than the big companies. But you can understand a specific situation better than anyone else.

You can understand better:

• when this user needs context;
• when they need a prompt;
• when something must not be executed automatically;
• what information is worth remembering;
• what information must expire;
• what results need their sources explained;
• what actions must be undoable.

None of that is model capability. It’s product judgment.

A more concrete example

Suppose you build an AI fitness assistant.

With model capability alone, it can answer:

“How do I do a squat?” “What should I eat to build muscle?” “Make me a training plan.”

But to become a truly useful agent, it needs to know:

the user’s training goals; physical limits; which exercises they’ve done before; how the last session went; which movements hurt; what this cycle focuses on; when to add weight; when to deload; when to suggest rest.

At that point, what really matters isn’t whether the model can write a training plan.

It’s whether the system can maintain a long-term, updatable state that doesn’t end up hurting the user.

Likewise, if you build an AI writing tool, having the model “write well” isn’t enough.

You need to remember the user’s style. The target audience. Which expressions not to use. Past posts. What positioning this account is building. And at the same time, avoid letting the style of one post contaminate every output.

So the hard part of AI products is less and less “getting the model to output a paragraph.”

It’s:

designing, around a real situation, a system that lets the model work reliably.

Finally

“This AI is really strong.”

That statement is too crude.

A more accurate set of questions would be:

Is the model strong, or the agent? Can it reason, or can it find material? Can it write, or can it execute? Does it remember, or was it just given good context this time? Does it really understand the task, or did the product layer pave the way for it?

This isn’t nitpicking.

It’s the most basic layer of taking an AI product apart.

The model is like the engine. The agent is like the whole car.

The engine matters. But the steering wheel, brakes, navigation, dashboard and seatbelt matter just as much.

An AI product with lots of horsepower and no brakes isn’t advanced. It’s terrifying.

So the truly strong AI of the future won’t just be “a smarter model.”

It’ll be:

a strong enough model; precise context; memory that doesn’t get mixed up; tools that can execute; a process that keeps moving; boundaries that can be controlled; results that can be verified.

Only then is it more than a system that answers questions. It starts to become a system you can really work with.