LLM / Text API Pricing

How LLM token pricing actually works

A token is not a word, a sentence, or a character — it's a variable-length fragment of text that roughly maps to 0.75 words in English. Every LLM API bills you for two separate token counts: the tokens you send (your prompt, system instructions, conversation history, any documents you include) and the tokens the model generates back (the completion). Output tokens are almost always priced higher than input, often 5–6x higher, because text generation is computationally more expensive than text encoding. A concrete example: at $5 input / $30 output per million tokens, a 500-token prompt that generates a 1,000-token response costs $0.0025 in + $0.03 out = $0.0325 per call. At 10,000 calls a day, that's $325/day. The output multiplier is where most production LLM bills come from.

Prompt caching: the highest-leverage cost optimization

If your application sends the same content in every API call — a system prompt, a knowledge base document, a list of instructions — prompt caching can reduce that repeated context from full input price to a fractional cache hit price. OpenAI's cached input pricing drops input costs by roughly 90% on cache hits; Google's context caching on Gemini 3.1 drops input to $0.20/M (from $2.00/M — also a 90% reduction) for content that's been stored in the cache. Anthropic offers a similar mechanism on Claude. This single optimization typically has a larger impact on production LLM costs than switching model providers entirely. If you're running a chatbot, agent, or any application that prepends a large fixed prompt on every call, prompt caching should be the first thing you implement — not switching to a cheaper model.

When to use mini/budget model variants

Every major provider now offers at least one "mini" or budget variant of their flagship model family: GPT-5.4 Mini, Gemini 3.1 Flash-Lite, Claude Haiku. These are typically 5–15x cheaper per token and cover a surprisingly large fraction of real production use cases: classification, routing, summarization, FAQ answering, form extraction, and simple Q&A. The mistake most teams make is defaulting to the flagship model for all tasks and only switching to the cheaper variant for a few known-simple use cases. A better default is to use the mini model first and escalate to the flagship only when you hit a capability ceiling. This inversion in default behavior usually cuts LLM costs 60–80% for production workloads without measurably degrading output quality for the majority of requests.

Long-context pricing tiers and the context window trap

Several models bill at a higher rate once your input exceeds a threshold — Gemini 3.1 Pro, for instance, doubles its input price for requests over 200K tokens ($2.00 to $4.00/M input, $12 to $18/M output). If your application passes large documents or long conversation histories, check whether the model you're using has a long-context pricing tier, and at what threshold it kicks in. A naive assumption of flat per-token pricing on a 500K-token context window can produce a real bill that's double the estimate. The practical fix is to summarize or truncate context where possible rather than passing raw history, and to use context caching for repeated fixed content rather than re-sending it on every call.