Do the cheap work first

The first version of any document-processing feature looks the same. Take the file, send it to a model, ask for structured output back. It works on the first try, which is the genuinely new thing about building with these models, and it is why the first version always looks like that.

Then you run it over a few hundred real files and the bill arrives.

The instinct at that point is to go shopping — a smaller model, a cheaper tier, a different provider. Sometimes that's right. But it's worth doing in the other order, because the model is usually not where the money went. The money went to work that didn't need a model at all.

Three questions before the model

Does this need a model?

A surprising amount of what gets sent to a language model is deterministic work wearing a costume.

Finding where a section starts. Reading a page header. Pulling a table out of a layout that already describes its own cells. Cropping a figure. These are all solvable with a parser and an afternoon, they run in milliseconds, they cost nothing per call, and — the part that matters more than cost — they give the same answer every time.

A model asked to find a section boundary will get it right almost always, which sounds fine until you're debugging the almost. Deterministic code that's wrong is wrong reproducibly, and you can fix it. A model that's wrong 2% of the time is a different kind of problem.

So the first pass over a document should be the cheap one: parse what's parseable, extract what's structured, and narrow down what's actually left.

Does anyone want this part?

The second question is the one that saved the most, and it's embarrassingly obvious in hindsight: are you processing content that the user has switched off?

Features get configured. A customer enables two of your four document types. A plan tier excludes a section. If your pipeline ingests the whole file and then filters the results, you paid full price for the parts you threw away.

Doing the filtering first means a cheap structural pass works out which ranges are in scope, and the expensive path only ever sees those. The code is barely different. The bill is not.

How many calls is this, really?

There's usually a choice between one pass per page and one pass per section, and the difference is an order of magnitude.

If the document already contains real text, you can batch generously — a whole section per call, a handful of calls for a long document. If it's a flat scan with no text layer, you're stuck going page by page through something that can read images.

The point is that this is a branch, and a lot of pipelines don't have it. They take the slow path for everything because the slow path is the one that always works. Detecting the easy case and handling it cheaply is often the single biggest win available, and it's one if.

Then make the calls cheaper

Only once the above is settled does tuning the calls pay off.

Stable prefixes. If every call starts with the same long instruction block, structure the prompt so that block is byte-identical across calls and comes first. Most providers can reuse work on a repeated prefix, and that only helps if the prefix really is repeated — interpolating a timestamp or a file name into your system prompt quietly defeats it. Check your provider's current docs for the specifics; they change.

Cap the output. Output tokens usually cost more than input. "Return JSON matching this schema" is cheaper and more useful than "describe what you see," and it's easier to validate.

Fail fast and loudly. A job that retries an expensive call forever is a cost incident, not just a stuck job. Cap the attempts, put a deadline on the whole job, and make sure two workers can't claim the same item — double-processing is double-billing.

Why this order

It's not really about money. The cheap passes are faster, they're testable without a network call, they're debuggable, and they shrink the surface where a model can be creative at your expense.

The bill is just the thing that makes you go looking.