Skip to content
All posts

What will this AI feature cost to run? A back-of-envelope method

AI features have unit economics hiding inside them. Here is a simple way to estimate cost per request and per customer before you build, and the levers that bring it down.

Shubham Somani 4 min read

Most software gets cheaper per user as you grow. AI features can do the opposite. Every request is a metered call to a model provider, and a feature that costs pennies in a demo can cost real money at scale, or in the hands of one enthusiastic customer.

You don’t need a finance team to see this coming. You need about fifteen minutes and a spreadsheet, before you write any code.

Step 1: Estimate tokens per request

Model providers charge per token, roughly three-quarters of a word in English, and they price input and output tokens separately. For one request, add up:

  • Input tokens: your system prompt and instructions, plus any context you retrieve (help articles, document chunks, past messages), plus the user’s input.
  • Output tokens: the length of the answer you ask for.

Retrieved context is usually the big one. Five document chunks of 500 words each is already around 3,000 input tokens before the user has typed anything.

Then count the calls. A feature that classifies, then retrieves, then drafts, then checks its own draft is four model calls, not one.

Step 2: Turn tokens into money

Look up the current per-million-token prices for the model you plan to use. They change often, so always use the provider’s pricing page, not a blog post. Then:

cost per request = (input tokens × input price + output tokens × output price) ÷ 1,000,000

Do this for each call in the chain and add them together.

Step 3: Multiply by real usage

Now estimate how often it runs:

monthly cost = cost per request × requests per user per month × active users

Do it three times, for a light user, a typical user and your heaviest user. The heavy user matters most. Flat-priced plans with unlimited AI usage are where margins quietly disappear.

Then compare the cost per customer with what that customer pays you. If the AI feature eats a large slice of the subscription price at typical usage, you have a pricing problem to solve before launch, not after.

Step 4: Add the costs people forget

  • Retries and fallbacks. Failed or invalid responses get retried. Budget for some fraction of calls running twice.
  • Evals. Running your eval set on every prompt change costs tokens too. It’s worth it, but count it.
  • Embeddings and storage for retrieval: usually small, but not zero.
  • Growth in context. Chat-style features resend the conversation history on every turn, so later turns cost more than early ones.

The levers that bring it down

If the numbers look uncomfortable, you have more options than “don’t build it”:

  • Use the smallest model that passes your evals. Routing, classification and extraction rarely need the biggest model. Save it for the steps that actually need it.
  • Retrieve less, better. Sending three relevant chunks beats sending ten loosely related ones, for both cost and quality.
  • Cache. Many requests repeat. Cache answers to identical inputs, and use the provider’s prompt caching for long, stable system prompts.
  • Cap output length. Ask for exactly the structure you need, not an essay.
  • Batch what isn’t urgent. Overnight summaries and back-office extraction can often use cheaper batch pricing.
  • Price for it. Usage limits, credits or an AI add-on tier keep heavy users from turning a good feature into a loss.

Track it like any other metric

The estimate gets you to a build decision. After launch, log tokens and cost for every request, and put cost per request and cost per active customer on the same dashboard as latency and quality. Alert when they move.

Model prices fall, new models appear and usage patterns change. A team that watches these numbers weekly can switch models or tune prompts with confidence. A team that doesn’t usually finds out from the monthly invoice.

The short version

Before building an AI feature: count tokens per call, calls per request, requests per user and users. Multiply, compare it with what the customer pays, and pull the cheap levers first. It’s boring arithmetic, and it’s the difference between an AI feature that grows your margin and one that quietly eats it.