My mistake building AI products: thinking about cost too late

Recently, while testing an AI product I built, I found that several features actually cost noticeably more than expected.

It’s an AI job-search product, covering job scanning, fit analysis, company research and generating application materials. Looked at one by one, none of the features seemed outrageous: just searching some information, analyzing a job posting, generating a report.

But once you string together one complete flow of use, the problem appears.

From finding jobs and reading the posting, to analyzing fit and researching the company, to generating application materials, one complete cycle could cost several dollars.

For an occasional test, the number seemed acceptable. But if every user costs several dollars per use, the cost balloons quickly as soon as users grow a little.

That’s clearly not sustainable.

One feature isn’t one model call

I hadn’t fully realized that one button users see on the page can hide a whole chain of paid calls.

For example, when a user clicks “research this company” once, the system may need to:

• search several sources;
• read different web pages;
• fill in missing information;
• analyze several dimensions separately;
• and finally generate a combined result.

If some step doesn’t finish, a continuation or retry may also happen.

On the surface it only generated a few hundred words, but the model may already have searched, read and processed a lot of content.

Another kind of hidden cost comes from repeated input.

To analyze one job, different steps may repeatedly read the same job description, the user’s career history, job preferences and the previous step’s results. The final output isn’t long, but the same background has been sent several times.

So the cost of an AI feature can’t be judged only by how many words it finally generated, nor by counting the buttons on the page.

What really needs looking at is how many calls happened in the whole process, how much each one read, which tools it used, and what happened again on failures and retries.

What made me stop was that the overall cost no longer made sense

In one test, two rounds of company research produced no usable result in the end, and the various execution stages added up to about 610,000 combined tokens.

That number can’t be converted directly into an accurate bill, because the records at the time didn’t fully separate input, output, cache and search content. But it at least showed one thing: even when the page finally shows “failed,” searching, web reading and model processing may already have happened.

Still, failures costing money was only part of the problem.

What really made me stop was that, looking back over the whole product, I found many capabilities had never been seriously costed:

• how many model calls one complete use actually takes;
• which steps really need AI;
• which content is being sent repeatedly;
• under what conditions automated tasks keep spending money;
• how much one search, analysis or generation should be worth at most;
• whether this setup still holds when users go from 1 to 100.

I had been judging more whether features made sense, whether results were useful, whether flows were complete, without weighing “what you get” against “what you pay” item by item.

Every capability actually has a price.

A full job analysis, a company research report, a tailored résumé and a short message are worth different amounts and need different costs. Just because AI can do all of them doesn’t mean calling it as fully as possible is best.

The later you think about cost, the less it looks like a small optimization

Once I found the problem, I originally thought what needed doing was counting tokens and adding a few limits to model calls.

Only once I really started did I find it went far beyond that.

First I needed to take stock of every feature:

• which operations generate paid calls;
• how many calls each makes;
• whether it uses search or web reading;
• whether there are automatic retries;
• whether costs can still be incurred after failure;
• which background information is sent repeatedly across steps.

Then I also had to redesign:

• which work goes to ordinary code;
• which features should only be triggered by the user;
• which information can be cached and reused;
• how much input, output and search each capability is allowed;
• how all paid calls are managed in one place;
• whether result quality holds up once costs drop.

Finally, every existing feature had to be migrated to the new setup one by one.

The work took several days, and paused the feature development I had planned.

The later you think about cost, the less what you add at the end is a cost-tracking page, and the more it’s a round of architectural changes across the whole product.

If I did it again, I’d do three things earlier

At product design time, write a cost assumption for each capability first

You don’t need to build a complex system on day one, but you should at least be clear about:

• whether this feature needs AI;
• whether it’s triggered by the user or runs automatically;
• how many calls one use is expected to make;
• whether it needs search or reading outside content;
• roughly how much a single result is worth paying for;
• how failures and retries are handled.

Cost should be discussed together with user value and feature scope, not tacked on after development.

When the second AI feature appears, build a unified call entry point

If every feature connects to models on its own, it’s very hard later to limit models, call counts, retries and budgets in one place.

When the second AI feature appears, start building a minimal unified entry point and usage records, rather than waiting until the product already has a dozen call chains to migrate one by one.

Before search, automated tasks and multi-step flows appear, set hard boundaries

As soon as a feature starts involving:

• web search or reading;
• automatic or scheduled runs;
• several model steps;
• background tasks and concurrency;
• retries or continuations;
• large files and outside content of unpredictable length;

costs may stop growing steadily and start swinging wildly.

At that point you must define limits on input, output, tool calls and cost per run in advance.

Finally

This experience made me realize that with AI products you can’t only ask:

can this capability be built?

You also have to keep asking:

what does each run cost? Is the value it brings worth that price? Once usage grows, does this trade still hold?

Cost doesn’t need to be designed into a huge system on day one.

But from day one, every AI capability should carry a rough price tag.