Playground
Price-Blind Research Auditor
Audit a research prompt bundle for price, direction, and position-state leakage — the contamination that biases LLM trade theses.
Runs in your browser. Nothing you enter is uploaded, and no account or API key is needed.
1. Paste research prompt or context bundle
Paste the system prompt, user prompt, and/or retrieved context the LLM will see. Every line is scanned for explicit prices, directional framing, and position-state leakage. Nothing leaves your browser.
Why price-blind research matters
When an LLM sees current prices, directional language, or position state, it reliably generates a thesis that rationalises whichever direction the data implies. The cleanest agent architectures keep research strictly price-blind: the LLM produces a probabilistic view before the risk engine compares it to market state.
Load the demo to see how a typical (contaminated) prompt bundle scores.
How to use it
- Paste the prompt bundle the model will see: system prompt, user prompt and retrieved context. A contaminated demo loads by default.
- Read the leakage score (0 to 100%, severity-weighted) and the verdict: clean, light, caution or contaminated.
- Go through the findings by line. Each shows the rule, the matched text and why it biases the model.
- Remove or rewrite flagged prices, quotes, % moves, high/low language and position or P&L metadata, then re-check the cleaned text.
- Re-run whenever the prompt template or the retrieval sources change. The scan is rule-based and runs in your browser.
Questions people ask
What does 'price-blind audit' mean?
Checking that the text an LLM sees before a research or trade decision carries no prices, recent moves, highs or lows, or position and P&L state, so the model cannot reason backwards from what the market already did. The auditor scans pasted text line by line with fixed rules; it does not read your trades or their outcomes.
Why does the methodology emphasize 'process over outcome'?
A model shown the outcome (a price spike, an open profit) tends to write a thesis that justifies it, which is hindsight bias built into the prompt. Removing outcome information keeps the model's reasoning on the inputs you control, which is the part of a research process you can judge.
How do I use the audit feedback?
Fix every high-severity hit first (explicit prices and quotes, % moves, new highs or lows, after-the-move framing, position and P&L metadata), then review the medium ones (chart-pattern language, stop and target references, recency next to a number). A single explicit price is enough to move the verdict to caution.
Does this work for systematic traders?
Yes, for any pipeline that builds an LLM prompt from data: paste a rendered prompt and the scan shows which template fields leak price or position state. It does not audit strategy code.
What's the calibration report?
There is none. The tool reports a leakage score, a verdict and per-line findings for the text you paste; it does not grade decisions or track outcomes. The Calibration Dojo covers forecast calibration.
Related tools
- Playgrounds Prompt Injection Tester
Red-team a finance agent against 23 documented prompt-injection attacks — direct override, role confusion, indirect injection via retrieved content.
- Playgrounds Hallucination Detector
Paste a source document + an LLM's extraction. Every numeric claim in the output is checked against the source. Client-side. Catches silent fabrication.
- Generators Trading System Blueprinter
Pick your data source, LLM, broker, storage, risk engine, and logger. Get a Mermaid architecture diagram and a copyable starter file tree — the full stack before you write code.
Articles
- 8 min read The 5 Failure Modes of LLM Trading Agents (2026)
The 5 recurring failure modes in retail LLM trading agents: price-blind leaks, numeric fabrication, prompt drift, token runaway, audit amnesia.
- 10 min read Prompt Injection Attack Catalog for Finance Agents
Prompt injection attacks on finance agents — indirect injection via news feeds, tool-result poisoning, prompt exfiltration, unit confusion — plus defenses.
- 10 min read Postmortem Template for LLM Trading Systems
A blameless, append-only postmortem template plus a 20-mode failure checklist — price-blind leaks to cache poisoning — keyed to the trace-ID log.
Workflows that use this tool
- Workflow Audit your pipeline
Catch hallucinations, prompt injections, and regression drift before they ship.
Use it from code
The same calculation as a JavaScript module you can import. It runs where you import it, with no request, key or rate limit.
import { compute } from "https://aifinhub.io/engines/price-blind-auditor.js";