Every AI API bills by the token, and every token you send also uses up context window and adds latency. The biggest savings rarely come from shaving words off a sentence. They come from not re-sending the same large context at full price (prompt caching), not sending irrelevant context at all (retrieval and chunking), and limiting output length, which is priced several times higher than input.
This guide covers those three levers first, then the prompt-writing habits that trim the rest. New to how tokens and context windows work? Start with What Is an LLM?.
How Tokens Are Counted and Billed
- A token is a chunk of text, roughly 3–4 characters of English on average. Code, non-English text and unusual formatting usually take more tokens per word.
- Input and output are priced separately. Output typically costs 4–5 times as much per token. On Claude Sonnet 5, for example, input is $2 and output is $10 per million tokens.
- APIs are stateless. In a chat, every new turn resends the whole conversation so far, so a long conversation gets more expensive with each message. Saying “use the summary from before” does not avoid this; the earlier messages are still sent.
- Each provider uses its own tokenizer, so measure with that provider’s tools rather than a rule of thumb. Anthropic has a token counting endpoint (
POST /v1/messages/count_tokens), and every API response reports the exact input and output tokens used.
Lever 1: Prompt Caching
If many requests start with the same long prefix, such as a system prompt, tool definitions, reference documents or a codebase summary, prompt caching lets the provider store it and charge a fraction of the normal price when it is reused.
| Claude API price vs normal input | |
|---|---|
| Cache write (5-minute lifetime) | 1.25× |
| Cache write (1-hour lifetime) | 2× |
| Cache read | about 0.1× |
Worked example on Claude Sonnet 5: a 8,000-token system prompt plus a 500-token question, 10,000 times a day. Without caching that is about $170/day in input tokens. With the system prompt cached (and requests arriving often enough to keep it warm), it is about $26/day. Same prompt, same answers, roughly 85% less.
- Put stable content first and changing content (the user’s question, timestamps, IDs) last. Caching matches the prefix exactly, so a timestamp at the top of your system prompt silently breaks it.
- Mind the minimum. Short prompts are not cached at all (1,024 tokens on Claude Sonnet 5, and it varies by model).
- Check it works: the response usage shows cached tokens read (
cache_read_input_tokenson Claude). If it stays at zero across repeated requests, something in your prefix is changing. - OpenAI and other providers offer their own prompt caching with different rules and discounts; check each provider’s docs. Local and hosted open-model platforms often price cached input far lower too: on Ollama Cloud, cached input for
deepseek-v4-procosts $0.044 per million tokens versus $1.32 uncached.
Lever 2: Send Only Relevant Context
- Retrieve, don’t paste. Instead of sending a whole manual, split it into chunks, embed them, and send only the few chunks relevant to the question. That is retrieval-augmented generation, and it often cuts input by an order of magnitude.
- Summarize long inputs once. Condense a long transcript or document in one call, store the summary, and use the summary in later calls.
- Trim conversation history. In long chats, drop or summarize old turns rather than resending everything forever.
- Scope code requests. “Check
auth/login.pyfor SQL injection” beats “review this entire codebase for security issues”, in both cost and answer quality.
Lever 3: Control Output Length
Because output tokens cost more, asking for exactly the output you need is one of the cheapest wins:
- Ask for a format and length: “3 bullet points”, “a JSON object with fields name and score”, “one sentence”.
- Set a sensible
max_tokenslimit so a runaway answer cannot cost more than you expect (but not so low that answers are cut off). - Ask for just the changed lines or a diff instead of a whole rewritten file.
- For reasoning models, lower the reasoning or “effort” setting on simple tasks; hidden reasoning tokens are billed too.
Prompt-Writing Habits That Save Tokens
Remove repetition
Before: Please write me a professional email. I want the email to sound professional and respectful. Make sure it’s polite and professional.
After: Write a polite, professional email.
Be specific instead of long
| Vague (and often longer) | Specific |
|---|---|
| Analyze the following report, provide investment advice, market trends, suggestions for diversification, and potential risk factors. | List the 3 biggest risks in this report, one sentence each. |
| Explain quantum entanglement in a way I can understand as a high school student, with examples and analogies. | Explain quantum entanglement to a high school student with one analogy. |
Use templates in your code
In an app, keep prompts as templates with variables, such as Summarize {text} in a {tone} tone. Templates keep your stable instructions identical across calls, which is exactly what prompt caching needs.
Structure beats prose
Short labeled sections or a small JSON spec ({"task": "summarize", "tone": "neutral", "max_words": 200}) are both shorter and easier for the model to follow than a paragraph of instructions.
What Not to Cut
- Examples that define the format. One good example often saves a retry, and retries cost far more than the example.
- Constraints that matter, such as “do not change public function names”.
- Context the model genuinely needs. A cheaper request that gives a wrong answer is not cheaper. Judge cost per completed task, not per request.
Quick Checklist
- Is a large, stable prefix cached? Is anything volatile at the top of it?
- Am I sending only the context this question needs?
- Have I asked for a specific output format and length?
- Is conversation history growing without limit?
- Am I measuring real token usage from the API response rather than guessing?
Frequently Asked Questions
Rewriting or trimming a prompt so it uses fewer tokens while keeping the information the model needs to answer correctly.
Not if done carefully. Removing filler, redundant context and verbose phrasing usually keeps quality the same while cutting cost and latency.
Hand-optimization typically cuts 20 to 50 percent of input tokens. Using retrieval instead of pasting whole documents can save far more.
Drop politeness and preamble, replace long instructions with short rules, use bullet points over prose, and send only the relevant excerpt rather than the full document.
Pricing and caching figures checked against Anthropic’s API documentation and Ollama’s pricing page on September 25, 2026.