01
Read the units first
Character and Markdown modes count Unicode graphemes; estimated-token mode uses a published weighting formula. Size and overlap always share the displayed unit.
AI / LLM Dev Tools
RAG text chunker
Paste text, choose a unit, and inspect every source boundary before exporting.
RAG text chunker
Paste text or load the example to begin.
Input stays in this browser. No upload or network request is made.
Fixed windows count grapheme clusters and may cut through a sentence.
Paste text or load the example to begin.
A RAG text chunker is useful when a retrieval result loses the sentence just before a boundary. Paste a real passage here and compare where the cuts land before indexing it.
This text chunking visualizer shows chunk size and overlap as source ranges. It supports Markdown chunking and exports JSONL chunks with offsets, so a retrieved passage can be traced back to the original text.
Why use this RAG text chunker
Inspect the repeated text instead of guessing what overlap means.
Trace each cut back to the source
The RAG text chunker records half-open UTF-16 offsets and highlights the portion shared with the preceding chunk. A range points to the exact original text, including line endings.
Keep Markdown topics separate
Heading boundaries begin new chunks with no cross-section overlap. Long sections fall back to paragraphs and lines, so a new heading is not prefixed with the previous topic.
Tune a split you can verify
Three details that affect the chunk list more than a generic preset.
02
Compare the actual shared span
Select adjacent chunks to inspect their source offsets and highlighted overlap. Markdown headings intentionally reset overlap to zero.
03
Export after the result updates
Automatic splitting waits 350 ms after an edit. Turn it off for manual runs with Ctrl/Cmd + Enter; copying and downloading remain disabled while results are stale.
Boundaries, units, and export data
The output is designed for inspection and downstream indexing experiments.
Grapheme-safe character windows
Fixed windows do not cut a joined emoji or combining mark in half; they may still split a word or sentence.
Transparent token estimate
The estimate weights CJK, other scripts, Latin text, and symbols differently, without claiming to match any model tokenizer.
Complete JSONL export
Every JSONL line includes the chunk text, source start and end, character count, estimated tokens, and actual overlap range.
What the preview can and cannot prove
These limits belong to this splitter, not to a model or embedding API.
- Estimated tokens are not model tokens
- The weighted count is a planning aid. Check a model’s own tokenizer before using a hard input limit.
- Markdown is structural, not semantic
- Only ATX headings outside fenced code start sections. Oversize sections can still split a code block or table when the size limit requires it.
- Overlap must be smaller than size
- The page rejects invalid values instead of silently clipping them. Markdown starts a new section without carrying text across its heading.
- No partial files or retrieval score
- Input above 100,000 UTF-16 units or more than 2,000 chunks fails without an export. No embedding, vector database, or retrieval evaluation runs here.
RAG text chunker questions
Answers based on the behavior you can inspect on this page.
What chunk size should I use?
There is no universal size. Start with the 1,000-grapheme window, then compare smaller and larger settings against real retrieval questions and document structure.
Does my document leave the browser?
No. The RAG text chunker runs in this tab, and the pasted text is not uploaded or stored by this tool.
How do I chunk a PDF or a Word document?
Extract its text first, then paste the resulting plain text or Markdown. This page does not parse PDF or Word layout, so inspect tables and reading order before splitting.
Is semantic chunking better than fixed-size splitting?
It depends on the task. This page provides deterministic windows and Markdown heading boundaries; it does not calculate embeddings or semantic similarity.
How do I tell whether my chunking is actually working?
Inspect each boundary and the repeated span here, then run retrieval tests with questions whose answers you know. The preview alone cannot measure recall.
Share a missing case
Try the RAG text chunker on your document
Paste a real passage, adjust one setting at a time, and export a full JSONL record of the boundaries you inspected.