AI Token Counter

What is a token?

A token is the unit AI models read and write. It is not exactly a word: a token can be a full word, part of a word, punctuation, whitespace, a number, or a code symbol. That is why a short paragraph, a spreadsheet export, and a JavaScript file with the same character count can all use different numbers of tokens.

Read the full definition in our glossary entry on tokens, or learn how context windows work.

Why token counts matter before you send a prompt

Every AI model has a context window: the total text it can consider at once, including your prompt, pasted documents, previous messages, and the answer it generates. If your input is too long, the model may ignore the end of the document, shorten its answer, or lose important context from earlier in the conversation.

Counting tokens before you paste a long document helps you decide whether to summarize first, split into smaller chunks, or remove irrelevant sections. This matters for legal drafts, research papers, code files, transcripts, email threads, and any workflow where missing a section can change the answer. Several tools on this site measure their limits in tokens too: the AI Watermark Remover, for example, quotes its per-document allowance in tokens for exactly the reasons above.

How big is each model's context window?

Context windows have grown fast: the current flagship models from OpenAI, Anthropic and Google all fit about a million tokens, roughly 750,000 words, in a single conversation. These figures come from the providers' own model documentation (checked August 2026), and the usage bars above use the same numbers.

Two details worth knowing. First, the window covers everything: your prompt, any pasted documents, the whole conversation so far, and the answer being written. Second, even million-token models cap their output far lower, so a model that can read a novel still answers in essay-sized pieces. The odd one out is Gemini 1.5 Pro, whose two-million-token window remains the largest ever shipped, paired with one of the smallest output caps.

How each provider counts tokens

All three major providers land on roughly the same arithmetic for English prose, about 4 characters per token, but they get there with different tokenizers, which is why the same text never counts identically across models:

• OpenAI uses its open-source tiktoken tokenizer, and its long-standing rule of thumb is 1 token ≈ 4 characters ≈ 0.75 English words.
• Anthropic does not publish its tokenizer; exact counts come from the API's token-counting endpoint. Newer Claude generations use updated tokenizers, so the same text can count slightly differently between Claude versions.
• Google states that for Gemini a token is about 4 characters and 100 tokens is roughly 60 to 80 English words, with a countTokens API for exact figures.

Tokens are not only text. Google publishes exact media rates for Gemini: an image up to 384 pixels costs a flat 258 tokens (larger images are tiled into 768×768 sections at 258 tokens each), video costs about 263 tokens per second, and audio about 32 tokens per second. So a one-minute video clip is roughly 15,800 tokens before you have typed a word, which is more than most essays.

What happens when a context window maxes out?

The context window is a hard budget, not a suggestion. Your prompt, every pasted document, the whole conversation so far and the answer being written all share one fixed pool of tokens, and the answer needs room too: an input that fills the window to the brim leaves the model nothing to reply with. What happens at the limit depends on where you hit it:

• Direct API calls are rejected. Send more tokens than the model accepts and the request fails with an error before anything is generated. You have to shorten or split the input yourself.
• Chat apps quietly forget. Most chat products keep the conversation going by dropping or auto-summarizing the oldest messages. The model does not warn you; it simply stops knowing things you told it an hour ago, which is why long sessions start contradicting their own earlier answers.
• Some tools read only part of your document. An oversized paste or upload may be silently truncated, so the model answers from the beginning of the text and never sees the end. If an answer ignores your final sections, this is usually why.

Quality also degrades before the hard limit: the fuller the window, the more the model has to divide its attention, and details buried in the middle of a huge input are the first casualties. That is why the planning table below recommends splitting long material even when it technically fits.

Why Claude Code burns through its context faster with large inputs

Users of coding agents like Claude Code often notice the context percentage dropping much faster on some sessions than others. That is not a bug; it follows directly from how the window works:

• Everything it reads stays in the window. The percentage measures how much of the model's context window the session occupies. Every file it opens, every command output, every diff and every message is kept so the model can refer back to it. Reading one 5,000-line file claims that slice of the window for the rest of the session, so a single large read can move the percentage by double digits in one step.
• Code tokenizes heavier than prose. Brackets, indentation, variable names and JSON logs break into 2 to 3 times more tokens per line than plain English, so 1,000 lines of source code cost far more than 1,000 lines of an essay.
• Tool output counts as input. Build logs, test failures and long stack traces are fed back into the window just like your own messages. A verbose test run can consume more tokens than the code being tested.
• Part of the window is spoken for from the start. The system prompt and tool definitions occupy a fixed share before you type anything, which is why a fresh session never begins at a full 100%.

So a session of short typed questions drains slowly, while one that reads big files and pastes long logs drains in big steps. When the window does fill, Claude Code compacts the conversation, summarizing the oldest parts to free space, which is the same forget-the-oldest trade-off chat apps make. The practical fix is the point of this page: paste the relevant part rather than the whole file, and count before you paste.

Token planning by content type

These estimates show why token counting is more than a technical detail. The larger the input, the more important it is to plan what the model should read.

How to reduce tokens without losing meaning

• Remove headers, footers, page numbers, navigation text, and repeated disclaimers from pasted documents.
• Replace long examples with one representative example unless the model needs every case.
• Summarize background information first with the AI Summarizer, then ask the model to work from the summary.
• Split large documents by section and ask focused questions instead of one broad request.
• Use clear, precise instructions. A short prompt with a precise goal often outperforms a long prompt with vague requirements.

Frequently asked questions

Related tools

After counting tokens, use the right tool for the next step in your workflow.