Prompt Token Counter
Count the tokens in a prompt exactly, in your browser, with a real BPE tokenizer — then see the cost across GPT, Claude and Llama models before you send it.
Your text
Tokens
GPT-4oPaste some text to count its tokens.
About the Prompt Token Counter
This LLM token counter runs a real BPE tokenizer in your browser. For OpenAI models whose encoding is published — GPT-4o, o1, GPT-5, GPT-4, GPT-3.5 — the number it gives you is not an approximation or a rule of thumb: it is the same token count the API will bill you for, produced by the same algorithm and the same vocabulary. The port is verified against tiktoken itself, token id for token id, on every fixture in the test suite.
That distinction matters because almost every token counter on the web is really a character estimator. Dividing by four is fine for English prose and wrong everywhere else: code and JSON run nearer three characters per token because of punctuation and indentation, long numbers are split into groups of three digits, and non-Latin scripts are far more expensive still — character for character, Japanese costs roughly four times what English does. If you are sizing a context window or a bill, a rule of thumb is the thing that makes you wrong.
Claude and Llama are handled differently, and the page says so on every surface. Anthropic has not published a tokenizer for Claude 3 or later, and Llama's vocabulary is not ours to redistribute. So those numbers are estimates: they carry a visible range, a badge, and a note explaining exactly what they were derived from. You will never see a bare number here for a model we cannot actually count.
Nothing you paste is uploaded. The tokenizer vocabulary is downloaded to your browser once and cached, and the counting happens locally after that — which is also why there is no rate limit and why you can paste a confidential prompt, disconnect your network, and watch it keep working.
- Exact counts for every OpenAI model whose encoding is published, verified against tiktoken token-for-token
- Estimates for Claude and Llama shown as a range with their derivation, never as a bare number
- Every token displayed as a chip, so you can see exactly where the boundaries fall
- The same text counted under both OpenAI encodings side by side, plus both estimate families
- Input cost per request at current prices, with the date those prices were read
- Optional chat-request framing, shown as an explicit line item rather than folded into the total
- Control tokens like <|endoftext|> are counted as ordinary text by default, and flagged when present
- Characters, words, bytes and characters-per-token alongside the count
- Load a file, or paste directly. Everything stays in the browser
How to use it
- Paste your prompt into the left pane, or use Open file for something longer.
- Pick the model you are actually going to call. The badge next to the selector tells you whether that model's count is exact or estimated.
- Read the big number. If it has a ≈ in front of it, expand the panel underneath to see the range and why.
- Turn on Show tokens to see where the boundaries fall. This is the fastest way to understand why a prompt costs more than it looks like it should.
- Press Compare to count the same text under both OpenAI encodings at once — useful when you are choosing between model families.
- If you are counting a chat request rather than raw text, tick the chat framing box and set the message count. The overhead is shown separately so you can see what it adds.
Real-world use cases
AI & LLM application developers
Check the exact token cost of a system prompt or few-shot example set before shipping it, catching a prompt that is unexpectedly expensive before it goes to production traffic.
Backend & API developers
Size a request against a model's context window ahead of time, instead of discovering the limit from a 400 error in production.
Data & ML engineers
Estimate the token footprint of a batch of documents before running them through a pipeline, when the cost of getting the estimate wrong is a very large, very real bill.
Product & AI engineering leads
Compare how the same prompt tokenizes across OpenAI's two encodings before choosing a model family, especially for multilingual products where the difference is largest.
Prompt engineers
See exactly where token boundaries fall in a prompt using the token chips, to understand why a small wording change moved the count more than expected.
Examples
English prose tokenizes at about four characters per token
The quick brown fox jumps over the lazy dog.
10 tokens — 44 characters, 4.40 chars/token (identical under both encodings)
This is where the divide-by-four rule comes from, and it is close to the only case it holds for. Everything below is why you should not rely on it.
The same sentence in Japanese costs four times as much per character
素早い茶色の狐が怠け者の犬を飛び越える。
20 tokens under o200k_base, 30 under cl100k_base — from just 20 characters
One token per character on the newer encoding and half again as many on the older one, against 4.40 characters per token for the English above. Vocabularies are trained mostly on English, so other scripts are broken into much smaller pieces. This is also the clearest case for choosing a model family on token efficiency rather than sticker price: the same text costs 50% more to send to GPT-4 than to GPT-4o before you compare their rates at all.
Indentation is billed, and minifying is the cheapest saving available
{
"user": {
"id": 1
}
}16 tokens indented, 7 tokens minified — the same data for 56% less
Every level of indentation is real tokens, and pretty-printed JSON in a prompt is paid for on every single request. Both encodings agree here, so this saving is available whichever model you use.
A long number is not one token
1234567890
4 tokens
Digits are grouped in runs of at most three, so numbers are expensive relative to their length. A prompt full of IDs, timestamps or coordinates costs far more than its character count suggests.
An emoji can be one token or eighteen
🙂 versus 👨👩👧👦
🙂 is 1 token under o200k_base and 2 under cl100k_base; 👨👩👧👦 is 11 and 18
The family emoji is four people joined by invisible zero-width joiners, and every joiner is billed. Turn on Show tokens and it appears as a single chip with a dotted underline, because those tokens are fragments of one glyph rather than separate characters. If your product handles user-generated text, emoji are a real and easily-missed cost.
An invisible character costs a token and breaks the word
helloworld
3 tokens, where “helloworld” is 2
There is a zero-width space between the two words above. It is billed like any other character, and it also splits the word so a model matching on “helloworld” no longer sees it. The Prompt Cleaner finds and removes these.
Common errors
| Message | Cause | Fix |
|---|---|---|
| This number does not match what OpenAI billed me | The API bills for the whole request, not just your text: each message carries role and delimiter tokens, and tools, system prompts and prior turns are all input too. | Tick the chat framing box and set the number of messages, then add your system prompt and history as separate counts. The framing overhead is shown as its own figure so you can see the difference. |
| This number does not match Anthropic's count for Claude | It is not supposed to be exact. Claude's tokenizer is not published, so the Claude figure here is an estimate derived from a different vocabulary, and it is shown with a range for that reason. | If you need an exact Claude count, call Anthropic's count_tokens endpoint — that is the only source of truth. Use the figure here for sizing and comparison, not for invoicing. |
| My prompt fits the context window here but the API rejected it | The context window has to hold your input and the model's output. A 128k window with a 120k prompt leaves 8k for the answer, and reasoning models consume far more of that than they appear to. | Reserve output tokens explicitly rather than filling the window. Subtract your max_tokens from the window before deciding whether the prompt fits. |
| The token count jumped when I edited something trivial | Token boundaries depend on surrounding characters. Adding a space before a word usually changes which token the word becomes, because most vocabulary entries include a leading space. | This is normal and not a bug. Turn on Show tokens to see it happen — it is also why reformatting a prompt can change its cost without changing a word of it. |
| The page is downloading a megabyte | That is the tokenizer vocabulary. Counting tokens exactly requires the actual merge table, and there is no smaller honest way to do it in a browser. | It downloads once per browser and is then cached indefinitely, and it is shared across every AI tool on this site. Nothing is downloaded until you paste something, so a page view costs nothing. |
| It says it is only counting the first 200,000 characters | There is a deliberate cap, because the tokenizer runs on the same thread as the page and a multi-megabyte paste would freeze the tab. | The count shown is a real count of that prefix, and the page says exactly how much it used. For whole files above that size, count them in sections. |
Frequently asked questions
›Is this token count exact?
For OpenAI models whose encoding is published, yes — it is exact. This page implements the same byte pair encoding algorithm the API uses, with the same published vocabularies (cl100k_base and o200k_base), and the implementation is verified against tiktoken itself on every test fixture, comparing token ids rather than just counts. For Claude and Llama it is explicitly an estimate, shown with a range, because neither tokenizer is available to run.
›Why is the Claude count an estimate?
Anthropic has not published a tokenizer for Claude 3 or later. The vocabulary that shipped in older versions of their Python SDK is a GPT-2-era file for Claude 1 and 2 and gives wrong answers for current models. Anthropic does expose an exact count_tokens endpoint, but it needs an API key, so a page that runs entirely in your browser cannot call it. Rather than print a confident wrong number, this page counts your text with a published vocabulary and shows the result as a range with the assumption stated. If you need an exact figure, count_tokens is the answer.
›Why can't you count Gemini tokens?
Gemini uses a SentencePiece tokenizer, which is a structurally different algorithm from the byte pair encoding used by OpenAI and, as far as anyone can tell, by Anthropic. There is no published vocabulary to run and no sensible way to derive a Gemini count from a BPE count, so guessing one would be inventing a number. Google's API returns a countTokens value, which is the correct source.
›Is my prompt sent to a server?
No. The tokenizer vocabulary is downloaded to your browser once, and every count after that is computed locally. There is no API call, no logging and no rate limit, which is also the practical test: paste a prompt, disconnect your network, and keep typing. The count keeps updating.
›Why is the first count slower than the rest?
The first count for a given model has to download that model's vocabulary — 414 KB for cl100k_base or 966 KB for o200k_base, compressed — and build a lookup table from it. After that it is cached by your browser and shared with every other AI tool on this site, so the cost is paid once. Nothing downloads until you actually paste something.
›How many tokens is a word?
For ordinary English prose, roughly one token per three-quarters of a word, or about four characters per token. That ratio is unreliable everywhere else: code and JSON run closer to two or three characters per token because of punctuation and indentation, long numbers are split into groups of three digits, and non-Latin scripts frequently use a token per character or more. This is exactly why counting beats estimating.
›What is the difference between cl100k_base and o200k_base?
They are two generations of OpenAI vocabulary. cl100k_base has about 100,000 entries and is used by GPT-4, GPT-4 Turbo and GPT-3.5 Turbo; o200k_base has about 200,000 and is used by GPT-4o, o1, o3 and GPT-5. The difference is smaller than people expect and very unevenly distributed: on English prose, code, SQL, JSON and Markdown the two produce identical counts, and the extra 100,000 entries buy you almost nothing. Where they diverge is non-Latin script — in our own measurements o200k_base uses 41% fewer tokens for Russian, 58% fewer for Arabic, 33% fewer for Japanese and 39% fewer for a family emoji. So if your prompts are English and code, the encoding is not what to optimise; if they are multilingual, it may be the single biggest lever you have. Press Compare on this page to check your own text.
›Why do some tokens start with a space?
Because the vocabulary is built that way. Most common words appear as an entry that includes their leading space, so “ the” is a single token while “the” at the start of a line is a different one. This is why adding or removing a space can change a prompt's token count by more than you would expect, and why the token chips on this page show the whitespace rather than trimming it.
›What are those <|endoftext|> style tokens?
They are control tokens the model uses internally to mark boundaries, and each is a single token id despite being many characters long. If your text happens to contain one as literal text, this page counts it as ordinary text by default and tells you it found it, because that is what the API does unless you explicitly opt in. You can tick a box to count them as control tokens instead, and the number will change — which is the point of surfacing the choice rather than making it silently.
›Does the token count include the system prompt and tools?
Not automatically — this page counts the text you paste. A real API request also bills the system prompt, every prior turn of the conversation, and the JSON schema of any tools you pass, which for a tool-heavy agent can easily exceed the user's actual message. Count each part and add them, or use the chat framing option to include the per-message overhead.
Last updated