AI / LLM Dev Tools
Choose a real encoding, then inspect text tokens and IDs. Add your own template and capacity values when planning a request.
Text stays in this browser. Encoding data may download once; no prompt is sent to a model API.
Enter text to count.
Enter text to count.
A prompt can look short yet split into many tokens, especially when it mixes code, punctuation and several languages. This token counter exposes the actual IDs instead of guessing from character length.
Use the token counter before sending a batch of prompts. Each JSON string is measured as a separate request; the total describes workload, while the context check applies to each item.
The token counter uses the selected BPE vocabulary and reveals every token ID. A word with a leading space, a newline or a non-Latin character can split differently from the way it looks on screen. Byte fragments stay attached to their IDs even when one Unicode character spans several tokens.
A batch of prompts is not one giant context window. The token counter measures each JSON string independently, encodes its shared prefix and suffix with it, then shows the remaining room after your output reserve and estimated overhead. The batch total represents overall work, not a single request.
o200k_base is selected initially. Switch to cl100k_base only when that is the vocabulary you need. The token counter labels the actual encoding; it does not infer an arbitrary model's tokenizer from its name.
Batch mode accepts a JSON array of strings, so a newline inside one prompt remains inside that item. Counts refresh 350 ms after changes; switch Auto count off to run manually with Ctrl/Cmd+Enter.
Shared prefix and suffix are encoded together with each item. Their token difference can be negative when byte-pair merges cross a boundary. Enter a context window, output reserve and any known wrapper overhead yourself; stale results cannot be exported.
Use two implemented OpenAI BPE encodings rather than a characters-per-token shortcut. Token IDs and UTF-8 bytes make a tokenizer online inspection reproducible for the chosen vocabulary.
Count GPT tokens for up to 100 JSON string items without flattening multiline prompts. Show 50 items at a time, while the downloadable JSON keeps every item and every ID.
A shared prefix and suffix are joined to each prompt before tokenization. The tool reports source tokens, combined text tokens and their difference; it never assumes that separate token counts simply add.
Enter your own context window and reserved output. Each item's margin includes your optional overhead estimate; no model limit or price table is silently applied.
Different encodings have different vocabularies and merges. Switch between o200k_base and cl100k_base to inspect the difference for those two implemented encodings; another provider can differ again.
No. Four characters is a rough English prose average. Code, spaces, emoji and other scripts often split differently. Inspect the token IDs or paste the actual text instead of estimating from characters.
This token counter loads the BPE data into your browser and processes your text there. It does not send the prompt to a model counting endpoint; downloading the static vocabulary is separate from uploading text.
The API may include message boundaries, system text, tool definitions, attachments and generated output. This page measures only entered text and your explicitly entered wrapper estimate, so compare full-request usage at the provider.
No exact claim follows from an OpenAI encoding. These providers use other tokenizers. Use their own count endpoint for a precise request check; the ID list here belongs only to the chosen encoding.
Enter the intended window and output reserve. The context window calculator compares each item's combined text plus your overhead and reserve with that value. Verify the model's actual limit separately.
Unexpected count?
Use a representative text or JSON batch, check the ID boundaries, then enter the capacity values for the request you actually plan to send.