Playground
Price-Blind Research Auditor
Audit a research prompt bundle for price, direction, and position-state leakage — the contamination that biases LLM trade theses. Browser-only. Free.
- Inputs
- Paste + configure
- Runtime
- 1–15 s
- Privacy
- Client-side · no upload
- API key
- Not required
- Methodology
- Open →
1 · Paste research prompt or context bundle
Paste the system prompt, user prompt, and/or retrieved context the LLM will see. Every line is scanned for explicit prices, directional framing, and position-state leakage. Nothing leaves your browser.
Why price-blind research matters
When an LLM sees current prices, directional language, or position state, it reliably generates a thesis that rationalises whichever direction the data implies. The cleanest agent architectures keep research strictly price-blind: the LLM produces a probabilistic view before the risk engine compares it to market state.
Load the demo to see how a typical (contaminated) prompt bundle scores.
How to use
Step-by-step
- 1
Upload your trade log. The auditor strips outcome data and shuffles trade order.
- 2
Review each entry/exit decision in isolation. Grade decision quality (A/B/C/F) and tag your reasoning.
- 3
Submit grades. The auditor un-blinds the outcomes and compares grade-vs-outcome.
- 4
Read the calibration report: do your A trades actually win at higher rate than C trades? Misaligned grading = self-deception.
- 5
Repeat monthly with new trades. Calibration improves with practice if your decision-quality criteria genuinely predict outcomes.
For agents
Use in an agent
Same math, same result shape as the UI above — as a static ES module. No HTTP request, no auth, no rate limit.
import { compute } from "https://aifinhub.io/engines/price-blind-auditor.js";Contract: /contracts/price-blind-auditor.json Full agent guide →
Glossary references
Terms used by this tool
Questions people ask next
FAQ
What does 'price-blind audit' mean?
Reviewing your trade decisions without seeing the resulting P&L. The auditor randomizes the order of past trades and removes outcome columns. You judge each entry/exit on its own merits — was this a good decision given what was knowable at the time? Decoupling decision quality from outcomes is a behavioral-bias-mitigation technique.
Why does the methodology emphasize 'process over outcome'?
Single trade outcomes are noisy. A good decision can lose; a bad decision can win. Evaluating only outcomes biases your future decisions toward whatever happened to work last time, which is hindsight bias. The auditor forces you to grade the process — the part you actually control.
How do I use the audit feedback?
Tag each trade with your decision-quality grade (A/B/C/F) and your reasoning. After 30+ trades, the auditor produces a calibration report: do your A trades actually outperform your C trades? If not, your decision-quality criteria need revision. If yes, your process is sound — just keep doing it.
Does this work for systematic traders?
Less directly, because systematic strategies don't have per-trade discretionary choices. The auditor is more useful for discretionary traders or for evaluating regime-detection / strategy-selection decisions in a meta-systematic process.
What's the calibration report?
Cross-tab of self-assigned grade (A/B/C/F) versus realized outcome (winners/losers). Well-calibrated discretionary trader: A trades win at 65%+, F trades win below 35%. Miscalibrated trader: all grades win at roughly the same rate, indicating the grading criteria don't predict outcomes — usually a sign of confirmation bias in self-assessment.
Related deep dive
All articles →Read further
Long-form context behind the tool output.
- Methodology · Opinion·8 min
The 5 Failure Modes of LLM Trading Agents (2026)
The 5 recurring failure modes in retail LLM trading agents: price-blind leaks, numeric fabrication, prompt drift, token runaway, audit amnesia.
Read - Methodology · Opinion·10 min
Prompt Injection Attack Catalog for Finance Agents
Prompt injection attacks on finance agents — indirect injection via news feeds, tool-result poisoning, prompt exfiltration, unit confusion — plus defenses.
Read - Methodology · Opinion·10 min
Postmortem Template for LLM Trading Systems
A blameless, append-only postmortem template plus a 20-mode failure checklist — price-blind leaks to cache poisoning — keyed to the trace-ID log.
Read
Used in
Decision workflows that use this tool
Goal-driven flows that bundle this tool with adjacent ones.
Complementary tools
Users of this tool often explore
Prompt Injection Tester
Red-team a finance agent against 24 documented prompt-injection attacks — direct override, role confusion, indirect injection via retrieved content.
Hallucination Detector
Paste a source document + an LLM's extraction. Every numeric claim in the output is checked against the source. Client-side. Catches silent fabrication.
Trading System Blueprinter
Pick your data source, LLM, broker, storage, risk engine, and logger. Get a Mermaid architecture diagram, a starter repo scaffold (ZIP), and a list.