How to Set Up Prompt Caching for Finance

Identify the stable prefix

List every part of your prompt and label each as stable across calls or changing each call. System instructions, tool definitions, output schemas, and reference documents are usually stable; the specific market data, the user question, and the current step are usually variable. The stable parts are your cache candidates. In finance workloads the stable block is often the large majority of the tokens, which is why caching pays off so strongly here.

Anything you would copy unchanged into every request is a caching candidate. If you find yourself resending a long policy or schema each call, that is the prize.

Use the Token-Cost Optimizer: Compute the dollar cost of a trading research loop across Claude, GPT, and Gemini. Prompt length × model × retry × call volume → cost per idea and per.

Order the prompt stable-first

Caching covers the prompt only up to the first token that changes between calls. So place all the stable content at the front and all the variable content at the back. A single piece of volatile data near the top, such as a timestamp or the current price, breaks caching for everything after it. Reordering the prompt so the long stable prefix leads is the single most important step and often the only one needed.

Move dynamic values like the current date or quote out of the system prompt and into the trailing user message. One stray variable at the top forfeits the whole prefix discount.

Mark the cache breakpoint

Tell the provider where the cacheable prefix ends by marking a cache breakpoint at the boundary between the stable and variable content. The provider stores the prefix up to that point and serves it from cache on the next matching request. Place the breakpoint after the last stable block and before the first variable one. If your stable content has natural layers, you can mark multiple breakpoints so partial reuse still hits.

Mark the breakpoint at the largest stable boundary you can. The longer the cached prefix, the larger the per-call discount.

Keep calls inside the cache window

Cached prefixes expire after an idle period, so the discount applies only if you reuse the prefix before it expires. Workloads that fire many calls in quick succession, like an agent looping over markets, hit the cache reliably. Sporadic calls spaced beyond the window pay full price each time because the cache has lapsed. Batch related work together in time so the prefix stays warm across the run.

If your traffic is bursty, group requests that share a prefix into the same burst rather than spreading them across the day, so each burst pays the cache write once and reads cheaply after.

Measure cached versus uncached tokens

Confirm the cache is actually hitting by reading the usage breakdown the provider returns: it reports how many input tokens were served from cache versus billed at full rate, plus the one-time cache-write cost. A high cached-token fraction means your prefix is structured correctly. If the cached fraction is low, a variable token has crept into the prefix or the calls are spaced beyond the window. Measure before and after to quantify the saving.

The first call pays a small premium to write the cache; the savings come from the reads after it. Judge caching by the steady-state cost across many calls, not the first one.

Use the Agent Cost Envelope Calculator: Model an LLM research loop end-to-end — steps, tool calls, convergence checks, markets per day — and see per-loop, daily, and monthly cost with cost-cap.