TL;DR

After a year of retail LLM trading agents, five failure modes keep recurring — in forum post-mortems, in the rare public audits, and across strategy styles. They are: price-blind contamination (the model sees prices and retrofits a thesis), silent numeric fabrication (the model invents numbers that look sourced), prompt drift across model versions (a prompt that worked on Opus 4.5 silently degrades on 4.7), token-cost runaway (research loops that burn budget without closing on a decision), and audit-trail amnesia (the agent took a position but nobody, including the agent, can reproduce why). Each has a cheap detection layer you can run today.

Why a failure-mode catalogue matters

LLM agents for trading have no peer-reviewed incident taxonomy. Unlike classical algorithmic trading — where failure modes (slippage, flash crashes, latency starvation) have been catalogued for decades — the retail AI-agent stack is a year old. The patterns below are derived from publicly-discussed agent post-mortems, our own build notes, and the workflows the tools in this hub were designed to catch. Sharing them is an E-E-A-T signal in both directions: if you disagree with the classification, the methodology pages list the evidence.

1 · Price-blind contamination

Symptom. The agent consistently generates theses that rationalise the direction prices have already moved. You can tell because the conviction strength correlates too well with recent market moves, not with any underlying factor.

Mechanism. LLMs are powerful pattern matchers. Give a model a price, a percentage move, or chart-pattern language, and it writes the thesis that would make that price correct. The model is not lying; it is completing a pattern you seeded.

Detection. Run your prompt bundle — system prompt, user prompt, retrieved context — through the Price-Blind Research Auditor. The rules flag explicit prices, directional language ("rallied 4.7%"), and position-state leakage ("long from $430"). A leakage score above 0.2 means the agent is not doing what its name suggests.

Mitigation. Separate the research step from the risk step architecturally. The research agent sees fundamentals, earnings, and news — but never prices, quotes, or your position. The risk engine, which receives the research agent's probabilistic view, is the only component that sees market state.

2 · Silent numeric fabrication

Symptom. The agent extracts numbers from a 10-K, a conference transcript, or a pricing table. Most of the numbers are correct. A few are subtly wrong. There is no indication which.

Mechanism. LLMs are trained to produce fluent, authoritative output. When a document does not contain the precise figure requested, the model interpolates — it generates a number that looks plausible given the surrounding context. The output formatting is indistinguishable from correctly-extracted data.

Detection. Every numeric claim in an LLM output should be mechanically cross-checked against the source. The Hallucination Detector does this for pasted extractions; the same logic belongs inside any production pipeline that routes LLM output to a trading decision.

Mitigation. Treat LLM output as structured unverified input. Pass every extracted number through a verifier that scans the source document for that exact value (or a defensible neighbourhood). Reject the output on a miss; retry with a narrower prompt or downgrade to a classical parser for that field.

3 · Prompt drift across model versions

Symptom. A prompt that worked reliably on Claude Opus 4.5 starts behaving differently on 4.7 — subtly different outputs, occasional format breaks, drift in tone or length. Nothing overt enough to fail a smoke test; enough to destabilise a production agent.

Mechanism. Model upgrades change training data, safety tuning, and sampling behaviour. A prompt that exploited a specific behaviour of the older model may no longer get the same response. Most production agents were never regression-tested against multiple model versions; the problem surfaces as quiet quality decay.

Detection. Regression-test every production prompt across model versions before upgrading. The Prompt Regression Tester runs one prompt across Claude 4.5/4.6/4.7, GPT-5, and Gemini 2.5 with your own keys; it returns per-model outputs, length distribution, and a pairwise-agreement score.

Mitigation. Pin model versions in production. Upgrade deliberately with a regression suite; do not auto-follow provider defaults.

4 · Token-cost runaway

Symptom. A research loop enters a back-and-forth with tool calls that never closes. The bill surprises you at end-of-month. You cannot point to which run produced which trade.

Mechanism. Agent frameworks let the model call tools iteratively until it signals completion. If the completion condition is vague ("when you have a confident view"), the model hedges: it calls one more retrieval, reads one more filing, one more time. Retry logic compounds the problem — a transient tool failure doubles the cost of every step it occurs in.

Detection. Instrument every run. The Token-Cost Optimizer computes expected cost per idea and per validated trade given prompt size, model, and retry rate. Use it to budget before you launch; use the instrumented numbers to verify the budget held.

Mitigation. Hard-cap tool calls per run. Set an explicit max_steps and fail loudly on exceeding it. Log per-run token consumption to the same append-only store as your trade decisions; spikes are then queryable.

5 · Audit-trail amnesia

Symptom. The agent opened a position two days ago. Your post-mortem asks: what did it see, what did it infer, what sized it, and why? The answer is lossy. You have the final trade but not the context that produced it.

Mechanism. Retail agent builds conflate logging with storage. Logs go to stdout, trades go to a database, and model inputs/outputs go nowhere. Under scrutiny, only the final trade survives; the evidentiary chain is gone.

Detection. Audit any agent you operate by asking: for a specific closed trade, can you reproduce the exact prompt bundle, tool-call sequence, and model output that led to the decision? If the answer is not obviously yes, you have audit-trail amnesia.

Mitigation. Every agent action — retrieval, inference, tool call, trade — gets a row in an append-only log keyed by a run ID. The log is never mutated; post-mortems read against it. Cost trivially; catches every other failure on this list.

Why this matters now

Retail agents are moving from toy to production faster than the tooling around them. The five failure modes here are not theoretical — each has been observed, costed, and (if you read between the lines of forum posts) blamed on the wrong thing more than once. The tools on this site are the response we could build to that: each catches a narrow failure mode, cheaply, client-side, with no API key you did not already have.

If your agent is missing any of the five detection layers above, that is the cheapest next improvement you can make to it.

Tools referenced