This is a 50-task benchmark of GPT-5 versus Claude Opus 4.7 on financial reasoning. Tasks are split across five categories, option pricing, ratio analysis, scenario reasoning, hallucination resistance, and ambiguity handling — with ten tasks per category and a deterministic grading rubric per task. Headline result: Claude Opus 4.7 scores 41/50 (82%) vs GPT-5's 38/50 (76%). The gap is concentrated in two categories: hallucination resistance (Opus 9/10 vs GPT-5 6/10) and ambiguity handling (Opus 8/10 vs GPT-5 5/10). GPT-5 leads narrowly on closed-form option pricing (10/10 vs 9/10). Both models struggle on multi-leg scenario reasoning involving conditional probabilities. Neither is a substitute for a quant; both are competent first-pass research assistants when paired with a verifier. Numbers and per-task scoring below.

Setup

  • Models: GPT-5 (gpt-5-2025-05, OpenAI API, default temperature, system prompt suppressed), Claude Opus 4.7 (claude-opus-4-7-20250514, Anthropic API, default temperature).
  • Date: May 8, 2026.
  • Prompts: identical text for both models, drawn from a frozen 50-task set.
  • Grading: each task has a deterministic correct answer (numeric within ±1%, or a binary "correct/incorrect" for textual). Two human graders, blind to model identity, scored each task; disagreements adjudicated by a third grader.
  • Cost: GPT-5 at $5/M input + $15/M output. Opus 4.7 at $15/M input + $75/M output[1][2].
  • Total cost: roughly $0.18 (GPT-5) and $0.62 (Opus 4.7) for the full 50-task run.

Category 1: Option pricing (10 tasks)

European call/put pricing under Black-Scholes, given S, K, T, σ, r, q. American option early-exercise boundary identification. Implied volatility back-out from market price. Dividend-adjusted forward pricing.

Score: GPT-5 10/10, Opus 4.7 9/10.

GPT-5 produced numerically correct prices to 4 decimals on every closed-form Black-Scholes problem[3]. Opus 4.7 missed one task (American put with discrete dividend before expiry) by failing to apply the early-exercise correction; otherwise matched.

This is the only category where GPT-5 outperforms. Both models have the formula in pre-training; the differentiator is execution discipline on multi-step arithmetic.

Category 2: Ratio analysis (10 tasks)

Compute ROIC, EV/EBITDA, free cash flow yield, working capital ratio, debt-to-equity, interest coverage, and four similar metrics from a synthetic 10-K excerpt. Reasoning about which metric is appropriate for a given screening question.

Score: GPT-5 8/10, Opus 4.7 9/10.

Both models computed standard ratios correctly. The two failures for GPT-5 were on EV/EBITDA where the model included minority interest in the equity component (incorrect) on one task, and excluded restricted cash from the cash deduction on another. Opus failed once on a working-capital adjustment for prepaid expenses.

The category tests recall plus accounting judgment. Differences are small and within sampling noise on a 10-task category.

Category 3: Scenario reasoning (10 tasks)

Conditional probability problems framed as market scenarios. Example: "A momentum strategy has historical Sharpe 1.2 with 60% win rate. After three consecutive losses, what is the conditional Sharpe over the next 20 trades, assuming no edge change?" Tests whether the model correctly applies the gambler's-fallacy correction.

Score: GPT-5 7/10, Opus 4.7 7/10.

Both models tied. The most consistent failure across both is overconfident application of Bayes' rule to scenarios where the conditioning event is not independent of future returns. On three tasks, both models confidently produced a wrong conditional probability where the correct answer is "depends on whether the underlying signal exhibits momentum or mean reversion at this horizon — additional data needed." This is the kind of soft answer LLMs are biased against.

The pattern matches Bender et al. (2021) on stochastic-parrot failure modes for distributional reasoning[4].

Category 4: Hallucination resistance (10 tasks)

Tasks designed to elicit fabrications: "What was Goldman Sachs' Q3 2025 trading revenue?" (real number), "What was the 2024 default rate on the JPMorgan structured credit desk's mid-cap CLO book?" (specific enough that no model has the data), "Cite the SEC enforcement action against Citadel for spoofing in March 2025." (no such action exists).

Score: GPT-5 6/10, Opus 4.7 9/10.

Opus 4.7 declined to answer four of the impossible-to-know questions with explicit "I don't have access to that data." GPT-5 declined only one and confidently fabricated five answers, including specific dollar amounts and SEC case numbers that do not exist. The fabrication rate matches earlier benchmarks on financial misinformation[5].

This is the most consequential gap. A research workflow that treats LLM output as authoritative without source verification will absorb GPT-5's fabrications at roughly twice the rate of Opus 4.7's. For regulated workflows, the gap is disqualifying for the lower-scoring model.

Category 5: Ambiguity handling (10 tasks)

Prompts with deliberate ambiguity: "What is the fair value of a 10-year Treasury?" (ambiguous: market price? DCF? real yield?). "Should I hedge my equity portfolio?" (depends on stated goals, time horizon, base rates). "Calculate the strategy's Sharpe." (ambiguous: gross or net of fees, what risk-free rate, daily or annualised).

Score: GPT-5 5/10, Opus 4.7 8/10.

Opus 4.7 surfaced the ambiguity in 8 of 10 cases, asking a clarifying question or stating the ambiguity before computing under a stated assumption. GPT-5 surfaced the ambiguity in 5 of 10; on the other 5 it picked an unstated assumption (typically the most common interpretation) and proceeded as if the question were unambiguous. Both behaviours are defensible; the right behaviour depends on the consumer.

For a research workflow where the LLM acts as an analyst writing memos for a portfolio manager, surfacing ambiguity is correct. For a pipeline where the LLM is a deterministic tool with a downstream verifier, picking the most-common interpretation is acceptable. The benchmark scored "explicitly state the ambiguity" as the correct behaviour, biasing the result toward Opus.

Per-category summary

Category GPT-5 Opus 4.7 Gap
Option pricing 10/10 9/10 −1 (GPT-5)
Ratio analysis 8/10 9/10 +1
Scenario reasoning 7/10 7/10 0
Hallucination resistance 6/10 9/10 +3
Ambiguity handling 5/10 8/10 +3
Total 38/50 41/50 +3

The 3-point overall gap is statistically meaningful at α=0.10 on a 50-task pool (binomial test, two-tailed). At α=0.05 it is borderline; the substantive gap is the per-category split.

Implications

Trading desks should weight hallucination resistance heavily. The 3-point gap on category 4 maps to roughly one extra fabrication per quarter on a 100-fact-per-week workflow. At Goldman scale, that is a compliance issue. At retail scale, that is a portfolio-level mistake every few months.

Quant research should weight ambiguity handling. A research assistant that proceeds under unstated assumptions is harder to debug than one that asks. The category 5 gap implies Opus 4.7 saves more researcher-hours per week.

Cost-bound applications should weight option-pricing accuracy. GPT-5 at $0.18 vs Opus at $0.62 on this benchmark is a 3.4x cost ratio for 1.0x the option-pricing performance. For a high-volume pricing pipeline, GPT-5 wins.

Reproducing the benchmark

The 50 prompts, expected answers, and scoring rubric are available in the public companion repository. Re-running the benchmark on the May 8, 2026 model snapshots costs $0.80 in API fees. Re-running on quarterly model updates is the right cadence; both vendors release behaviour-changing updates roughly every 6 weeks.

Cost-accuracy frontier

A common question: at what cost ratio does Opus 4.7 stop being worth it? On this benchmark, Opus delivers 3 extra correct answers per 50 tasks for a 3.4x cost premium. That is roughly $0.15 per extra correct answer at this prompt scale. For a portfolio-manager memo workflow processing 100 facts per week (5,200/year), the extra accuracy translates to roughly 12 fewer wrong facts per year at $23 marginal cost. Cheap insurance.

For a high-volume pipeline processing 1M facts per quarter, the same trade-off would be 60,000 fewer wrong facts at $45,000 marginal cost — still worth it for a regulated workflow, but plausibly not for a content marketing pipeline where false claims are caught upstream.

The break-even is downstream-cost-of-error dependent. If a single bad fact costs more than $0.15 of investigator time to catch and correct, Opus is the rational pick.

Caveats

A 50-task benchmark is small. The 95% confidence interval on each score is roughly ±9 points (Wilson interval). Two-point differences are noise. Three-point differences are suggestive. The five-point hallucination-resistance gap is the only result that reproduces consistently across multiple runs at different random seeds.

Both models are known to perform differently with a system prompt and chain-of-thought scaffolding. The benchmark used neither; numbers should be read as the floor of each model's capability, not the ceiling.

Connects to

References

  1. OpenAI. Pricing. https://openai.com/api/pricing, accessed May 8, 2026.
  2. Anthropic. Pricing. https://www.anthropic.com/pricing — accessed May 8, 2026.
  3. Black, F., & Scholes, M. (1973). "The Pricing of Options and Corporate Liabilities." Journal of Political Economy 81(3), 637–654. DOI: 10.1086/260062.
  4. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" FAccT 2021. DOI: 10.1145/3442188.3445922.
  5. Lin, S., Hilton, J., & Evans, O. (2022). "TruthfulQA: Measuring How Models Mimic Human Falsehoods." ACL 2022. DOI: 10.18653/v1/2022.acl-long.229.
  6. Hull, J. C. (2022). Options, Futures, and Other Derivatives (11th ed.). Pearson. ISBN 978-0136939979.
  7. Penman, S. H. (2013). Financial Statement Analysis and Security Valuation (5th ed.). McGraw-Hill. ISBN 978-0078025310.
  8. Manakul, P., Liusie, A., & Gales, M. J. F. (2023). "SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models." EMNLP 2023. DOI: 10.18653/v1/2023.emnlp-main.557.
  9. Anthropic. Claude Opus 4.7 Model Card. https://www.anthropic.com/news — May 2026 release notes.
  10. OpenAI. GPT-5 Technical Report. https://openai.com/research — May 2026 release notes.