How technical does an AI PM need to be?

This is the second post in the series “Who is hiring AI PMs right now.”

For this round I collected 43 public AI product roles as of August 20, 2026, including positions at OpenAI, Anthropic, Scale AI, Box, Binance, Apple, Ford, n8n and companies in finance, insurance and healthcare, with reference to sources like the Stanford AI Index 2026 and PwC’s 2026 AI Jobs Barometer.

The first post drew a map:

so-called AI PM is already many completely different jobs.

Some are close to the model, some build agent platforms, some own evals, and some are responsible for bringing AI into specific industries and enterprise workflows.

That also explains a question I’d always found hard to answer:

how technical does an AI PM actually need to be?

If you only follow the discussion, you easily get two extreme answers:

one side says PMs don’t need to code; understanding users and products is enough;

the other says that to be an AI PM now, not knowing Python, ML and system design won’t cut it.

After going through this batch of real JDs, I think both sides are partly right.

The problem is:

“technical ability” isn’t one thing.

I’d roughly break the requirements in job postings into four layers.

Layer one: AI literacy — at least knowing what you’re doing

This is the most basic layer.

You don’t necessarily have to code, but you at least need to understand:

• roughly what LLMs are good and bad at;
• why hallucination happens;
• what problem RAG solves;
• how an agent differs from an ordinary chatbot;
• what tool calling, grounding, context and HITL each do.

These concepts aren’t hard to learn.

What really matters is whether they affect product judgment.

For example, an engineer says:

this scenario isn’t suited to stuffing more information into the context; it should switch to retrieval.

Or:

this step can’t be executed by the agent directly; better keep human approval.

The PM doesn’t necessarily have to implement it, but should at least be able to join the discussion:

why? What does it cost? Is there another option?

If every AI-related decision can in the end only become:

engineering recommends doing it this way.

then it isn’t only the technology being outsourced.

Part of the product judgment has been outsourced with it.

So I now understand AI literacy as:

you don’t need to become an engineer, but you at least need to understand why the product works the way it does.

That should already be the baseline for an AI PM.

Layer two: builder fluency — can you try it yourself first?

One layer up, I see more and more JDs starting to mention:

hands-on, prototype, Python, SQL, notebooks…

It’s easy to feel pressure here:

do PMs have to code now too?

I don’t think you need to jump straight to that conclusion.

What these roles more often express is a different requirement:

don’t depend on the engineering team for all validation.

For example, with a product idea, you could first, yourself:

• call a model API;
• set up a simple RAG;
• have an agent call two tools;
• run a few dozen test cases;
• compare two models;
• look at traces and logs;
• look at the results with some simple Python / SQL.

Box’s Product Manager, AI Evaluations is a very typical example.

It explicitly requires the PM to use Python, SQL and notebooks to build evals themselves, analyze results and prototype data pipelines.

That isn’t asking the PM to maintain production code.

It’s more like saying:

if a question can be validated yourself in a few hours, don’t first spend days writing documents and then wait for someone else to validate it for you.

This also has to do with the development of coding agents and AI-assisted building.

The cost of prototyping has dropped noticeably.

Some things for which “the PM can’t do it themselves” used to be perfectly reasonable have now become:

“why not just try it yourself first?”

So what really matters about builder fluency isn’t “do you know some programming language.”

It’s:

can you independently push a fuzzy idea to the point where there’s evidence to judge by?

Layer three: system fluency — knowing why an AI product fails

One layer further up is, I think, where AI PMs really start to separate.

A demo running doesn’t make it a product.

Once it’s really in use, you quickly run into:

• which tools can the agent call?
• which tools must it absolutely not call?
• how much should go into the context?
• when is memory needed?
• when a tool keeps failing, retry, fall back, or stop?
• which actions must a person confirm?
• after a model update, how do you know whether there’s a regression?
• quality goes up a little but cost doubles — ship it or not?
• the stronger model is two seconds slower — will users accept that?

At this point what you need to understand is no longer:

“what is RAG?”

But:

under what conditions the whole system is reliable, and where it breaks down.

For example, Binance’s AI Agent & Harness PM has responsibilities including:

task flow, capability boundaries, failure fallback, harness evaluation, task sets.

OpenAI’s API Agents PM needs to discuss SDKs, APIs and agent infrastructure together with research and engineering.

These roles don’t necessarily require the PM to implement everything.

But they require the PM to make product judgments about these things.

I think there’s an important distinction here:

layer two asks:

can I run experiments myself?

Layer three asks:

do I know what the experiment results mean, and whether this system can go live?

The latter matters far more than “being able to write a bit of Python.”

Layer four: ML / infrastructure depth — some AI PMs are technical PMs to begin with

Further up still are roles in core models, training, post-training, inference, compute and ML platforms.

They may require understanding:

• model training;
• inference;
• search / recommendation;
• data infrastructure;
• serving;
• distributed systems;
• experimentation frameworks.

Here, the competition often comes from:

ML engineers, data scientists, search / recommendation PMs, infrastructure PMs.

So if you look at some OpenAI or Anthropic JDs and think:

why does AI PM already require such deep technology?

it doesn’t necessarily mean all AI PMs have become like this.

A more accurate reading is:

you’re looking at a kind of PM that was always very close to the model and the infrastructure.

A healthcare agent PM and a core models PM may both have Product Manager in their titles, but their skill structures are completely different.

That’s also why classifying AI PMs first, in the last post, matters.

Otherwise every technical requirement gets mashed into one terrifying study list.

What I really think matters more is three abilities appearing across roles

After going through these JDs, there are three things I care about more than “can you code.”

1. Evals: how do you know it really got better?

This is where I think AI PM differs most clearly from traditional product work.

In the past, we could easily say:

“user feedback is good.”

In AI products, that isn’t quite enough.

You need to keep asking:

• compared with what baseline?
• better on which tasks?
• where did the test set come from?
• which errors improved?
• and where did it regress?
• which errors are acceptable?
• which errors must never happen even once?

So even if evals don’t become part of everyone’s job title, they’ll likely gradually become a basic working method for AI PMs.

Not everyone needs to become an eval expert.

But at least you should know:

on what basis am I saying this version is better?

2. Trade-offs: not simply chasing “a smarter model”

An AI product usually has many goals at once:

quality · latency · cost · reliability · safety

They often conflict with each other.

• a stronger model may be more expensive;
• a faster model may be a bit less effective;
• more context may be more accurate, but latency and cost rise with it;
• an agent with more permissions may complete more tasks, and also be more dangerous.

There’s no single answer an engineer can calculate here.

Product still has to decide:

what exactly are we optimizing for?

And:

where’s the floor?

3. Failure design: think first about how it will break

Ordinary software has to handle exceptions too, of course.

But the uncertainty of AI systems makes failure a more central product question.

• What if it doesn’t know?
• What if information conflicts?
• What if a tool returns an error?
• What if the model answers wrong with great confidence?
• What if execution fails halfway?
• When should a person take over?
• Can things be recovered after an error?

I now actually think:

one of the best ways to tell whether an AI PM really understands the product isn’t to see whether they can describe the happy path, but how they talk about failure.

So does an AI PM need to code or not?

If I have to give a very short answer:

not necessarily.

But the other answer is also very clear:

not touching technology at all will increasingly limit which roles you can compete for.

Not because PMs need to become engineers.

But because AI has turned many questions that used to be hidden in engineering implementation into product questions.

• which model to choose;
• when it can be released;
• how to evaluate quality;
• how much permission the agent has;
• what happens after a failure;
• how to trade off quality, latency and cost.

None of these can rely only on:

“the technical team will handle it.”

I think you can test yourself with five questions

Rather than asking yourself:

“do I know Python?”

take a real AI product and see whether you can answer:

1. Why does this need AI here, rather than ordinary rules?

2. What result counts as good?

3. How do you plan to test it?

4. Where is it most likely to fail?

5. After launch, how do you know whether it keeps getting worse?

If you can already answer these independently, then even with limited coding ability, technology has started truly serving product judgment.

If you can’t, then even knowing lots of frameworks and new terms, you may still only be:

knowing about AI, rather than knowing how to build AI products.

So after this research, my understanding of “how technical an AI PM needs to be” ended up a little simpler than when I started:

the point of learning technology isn’t to replace engineers.

It’s to avoid handing over, along with it, judgments that belong to product.

The next post will look at another very interesting change:

why are roles like Evals, Harness, Model Behaviors and Forward-Deployed, rarely seen a few years ago, starting to appear in job postings?

Are they just inventing new terms, or are AI product organizations really growing new specializations?

Part 2/4 of the series “Who is hiring AI PMs right now.” The research is based on 43 public AI product roles collected as of August 20, 2026; this post mainly draws on postings for Box AI Evaluations, OpenAI API Agents / Core Models, Binance AI Agent & Harness, Scale AI Forward-Deployed and others, cross-referenced with the Stanford AI Index 2026 and PwC’s 2026 AI Jobs Barometer for trends.