LLM Token

What Is LLM Token?

An LLM token is the unit a language model reads and writes text in: a short chunk of characters, on average about three-quarters of an English word. Tokens are also the billing unit for every model API, so token counts determine both what fits in a request and what each request costs.

Key Takeaways

• Rule of thumb for English: 1 token is roughly 4 characters, and 1,000 tokens is roughly 750 words. Anthropic's own documentation puts it at about 3.5 English characters per token for Claude models, with the exact ratio varying by language [1]. Code is denser in tokens because of symbols and whitespace.
• Every API bill is denominated in tokens, with output tokens typically priced several times higher than input tokens. Agent runs are usually input-heavy, which is why prompt caching matters so much.
• The token limit you hit in practice is the context window: the maximum tokens a model can process in one request, covering the prompt, the conversation so far, and the reply.
• Agents burn tokens fast. Every tool result, file read, and prior turn is resent on each step, so a single coding task can consume hundreds of thousands of input tokens.

How It Works

Models never see raw text. Before a request runs, a tokenizer splits the input into pieces from a fixed vocabulary, typically 50,000 to 200,000 entries learned during training, a process covered in more depth under tokenization. Common words like "the" get one token. Rarer strings split into fragments: "kubectl" might become "ku", "bect", "l". Each token maps to an ID, and those IDs are what the model actually processes. Generation runs the same way in reverse, with the model emitting one token at a time until it finishes.

Pricing follows mechanics. Providers meter input tokens (everything you send) and output tokens (everything the model generates) at separate rates, and those rates have collapsed: Stanford's 2025 AI Index found the inference cost of a GPT-3.5-level system dropped more than 280-fold per million tokens between November 2022 and October 2024 [2]. Reasoning models add a third bucket, thinking tokens, which you pay for as output even though they may never appear in the response. Because agent loops resend the growing transcript at every step, total input tokens grow roughly quadratically with conversation length. That is the single biggest lever on agent cost, and it is why teams summarize tool output, trim transcripts, and cache stable prompt prefixes.

Example

An engineer asks a coding agent to fix a failing test. The system prompt and tool definitions come to 6,000 tokens. The agent reads three source files at 4,000 tokens each, runs the test suite twice with 2,000 tokens of output per run, and works through eight reasoning turns. By the final step, each request carries the full accumulated transcript, and the run totals about 350,000 input tokens and 9,000 output tokens. With prompt caching on the stable prefix, most of those input tokens bill at a heavy discount, cutting the run from a few dollars to well under one. Nobody typed 350,000 tokens; the loop did.

What People Get Wrong

People treat tokens as words and estimate costs from word counts. The error compounds badly on real workloads. Code, JSON, URLs, and non-English text tokenize far less efficiently than English prose, sometimes at one token per character. A 2,000-word English document and a 2,000-word equivalent in Japanese or a JSON payload of similar visual length can differ by three times or more in token count. Measure with the provider's token counting endpoint instead of eyeballing length, especially before committing to a per-request budget.

FAQ

What is a token limit? The token limit, usually called the context window, is the maximum number of tokens a model can handle in one request, input and output combined. Current production models commonly offer 200,000 to over a million tokens. Exceed it and the API rejects the request or the oldest content gets truncated.

How many tokens is a typical word? In English, about 1.3 tokens per word on average. Short common words are a single token; long, rare, or technical words split into several. The ratio is worse for code and for most languages other than English.

Why do providers bill per token instead of per request? Compute cost scales with tokens processed, not with requests. A ten-word question and a hundred-page document are wildly different workloads, and token pricing tracks the actual GPU time each one consumes during AI inference. The volumes involved are staggering: Google reported processing over 480 trillion tokens a month across its products and APIs as of May 2025, up 50x from 9.7 trillion a year earlier [3].

Sources

1. Anthropic. "Claude glossary: one token approximates 3.5 English characters." https://platform.claude.com/docs/en/about-claude/glossary. Accessed August 2026.
2. Stanford HAI. "AI Index Report 2025: GPT-3.5-level inference cost fell over 280-fold per million tokens, November 2022 to October 2024." https://hai.stanford.edu/ai-index/2025-ai-index-report. Accessed August 2026.
3. Google. "I/O 2025 keynote: over 480 trillion tokens processed monthly, up 50x year over year." https://blog.google/technology/ai/io-2025-keynote/. Accessed August 2026.