LLM Cost Trends for Finance Workloads Statistics

The cost to query a model at GPT-3.5-level performance (64.8% on MMLU) fell about 280-fold, from roughly 20 US dollars per million tokens in November 2022 to about 0.07 US dollars per million tokens by October 2024

The AI Index defines the capability level by MMLU accuracy and tracks the cheapest model meeting it over time. The roughly 18-month window saw the price fall by more than two orders of magnitude.

Across benchmarks, LLM inference prices for a fixed capability have fallen between 9 and 900 times per year, with a median of about 50 times per year

Epoch measured price-to-reach-capability across six benchmarks including MMLU, GPQA Diamond, MATH, HumanEval, and Chatbot Arena ELO. The wide range reflects that some tasks commoditize far faster than others.

Restricting to data from January 2024 onward, the median annual price decline rose to about 200 times per year

Epoch notes the fastest declines (toward the 900x end) begin after January 2024 and cautions these rapid rates may not persist, so they should not be extrapolated linearly.

The price to match GPT-4's performance on a set of PhD-level science questions fell by about 40 times per year

This single-benchmark figure illustrates that even hard, reasoning-heavy tasks (the kind closest to financial analysis) saw order-of-magnitude annual cost declines, though slower than the easiest tasks.

Retrieval-augmented generation adds roughly 5 percentage points of accuracy and fine-tuning roughly 6 percentage points on a domain task, with the gains stacking

Cost-relevant because the cheaper-to-maintain option (RAG) captured most of the accuracy gain of the more expensive option (fine-tuning) in this study, informing build-versus-retrieve cost decisions for domain workloads.