LLM Reliability in Finance Statistics

A purpose-built benchmark measured hallucination rates of 5% to over 20% across leading models on finance-specific factual questions

Error rates varied widely by model and question type, with abstract conceptual questions and time-sensitive figures producing the most fabrications.

On the FinQA numerical-reasoning benchmark, the best published systems trailed expert human accuracy by a wide margin

FinQA pairs questions with earnings-report tables and text. The gap between model and expert performance highlights that multi-step financial arithmetic remains a weak point.

ConvFinQA extended financial QA to multi-turn conversations, where chained reasoning errors compounded across turns

Accuracy degraded as dialogues lengthened, because an early arithmetic mistake propagated into every dependent later answer.

Across general benchmarks, retrieval-augmented generation reduces but does not eliminate hallucination, with grounded systems still producing unsupported claims

The survey documents that grounding answers in retrieved context lowers fabrication rates yet leaves residual faithfulness errors, including citing sources that do not support the claim.

The IMF cautioned that opaque AI models in finance raise model-risk and explainability concerns that current governance frameworks only partly address

The report tied reliability and explainability gaps directly to financial-stability risk when models drive trading or credit decisions at scale.

Domain-adapted financial language models such as BloombergGPT outperformed comparable general models on finance tasks without sacrificing performance on general-language benchmarks, while still trailing far larger general models

Domain pre-training improved financial-task accuracy, confirming that finance reliability benefits from specialization but is not solved by it.