Token
A small piece of text, roughly one word.
The billing unit for AI: about 3/4 of a word, but not always.
Tokens are how models read
A token is a small chunk of text. Models don't read whole words or sentences at once. They read tokens: single characters, word pieces, or whole small words, depending on what's common.
"The" is one token. "Hamburger" is two tokens, "Hamb" and "urger". Even the space before a word is part of its token. Mathematical symbols, punctuation, and newlines also get tokenized. The pattern depends on the model's tokenizer, which is tuned during training.
Why tokens exist
Models are trained on numerical patterns, and tokens convert text into chunks manageable for math. Smaller chunks, like single characters, give you more precision but make sequences very long. Larger chunks, like whole words, are more efficient but miss detail. Tokens strike a balance, usually one to three tokens per word, optimized for the language and the model.
These are the real tokens computed by the o200k_base tokenizer (used by GPT-4o and o-series models). Each highlight is one token. Notice how a single word can span several colors, and how the leading space is part of the token.
Common short words are one token each, and the space in front belongs to the token.
The quick brown fox
loading tokenizer… · 19 characters
Less common words split into pieces: "tokenization" is " token" + "ization".
Understanding tokenization
loading tokenizer… · 26 characters
The rarer the word, the more pieces it shatters into.
Supercalifragilisticexpialidocious
loading tokenizer… · 34 characters
Type anything here to see how a model tokenizes it!
loading tokenizer… · 51 characters
Why tokens matter
- •Models charge per token. The word count is a rough guide at best.
- •A token is roughly 3/4 of a word on average in English
- •Rare or made-up words break into many small pieces
- •Every token in your prompt counts toward the cost
Why that question you asked cost 3 cents instead of 1.
Tokens: the unit of price
A token is about three-quarters of a word. "Hamburger" is three tokens, and "Hello" is one. Pricing is per 1,000 tokens. So if you are charged $2 per 1K tokens, and you send 2,000 tokens, you pay $4.
Input vs output: the split
Input is what you send, meaning your prompt, your documents, and your question. Output is what comes back, meaning the model's answer. Most models charge differently for each.
Input is cheaper because generating new text is harder than reading it. A 10-token prompt might cost 1 cent. A 100-token answer might cost 2 cents.
This is a pricing calculator for one conversation turn. You can pick a scenario, or you can drag the sliders to set your own sizes. The meter counts every token you send and every token that comes back, and it charges each side at its own rate.
You ask one short question and the model gives a short answer.
What you send
20 input tokens (about 15 words)
What comes back
160 output tokens (about 120 words)
The receipt
| Input: 20 tokens at $2 per million | $0.000040 |
| Output: 160 tokens at $8 per million | $0.001280 |
| Total for this turn | $0.001320 |
The meter runs on both sides of the conversation, and each output token costs 4x what an input token costs. The expensive tokens are the ones the model writes. You can compare the long essay to the document summary and see how the smaller reply keeps the bill down even when the prompt is huge.
How to estimate costs
Count tokens with the model's tokenizer tool. Multiply your input tokens by the input rate and your output tokens by the output rate, then add them up. A typical question and answer, where you ask 500 tokens and get back 200 at standard pricing, costs cents.
Check the comparison table for current rates. Most flagships charge $1 to $2 per 1M input tokens, and $3 to $5 per 1M output tokens.