Skip to content

Repository Token Estimator

Point the browser at a folder and get an estimated token map by directory and file type, plus how many context windows the codebase would fill.

Nothing is uploaded. Every file you select is read with FileReader inside this tab and never leaves it — there is no server call on this page, and the text is kept out of the URL as well, so the link you share carries your settings and not your content.

Estimated tokens in the files scanned
0

Choose a folder to scan. Folder picking works in Chrome, Edge and Safari; if your browser refuses, use the file picker instead.

Files read
0
Skipped by an ignore pattern
0
Skipped as binary
0
Skipped over 512 KB
0
Skipped at the scan cap
0
Bytes read
0
Estimated tokens
0
Bytes per token
Context windows to hold it all
Cost to send the whole thing once
$0

Estimated, not tokenized. Token figures here come from an approximation running in your browser: no tokenizer vocabulary is downloaded, because a real one is megabytes and nothing on this page fetches anything. It imitates how a byte-level BPE splits text — words, digit groups, punctuation runs, whitespace runs — but it has no merge table, so treat it as a planning number. The authoritative count is the usage object on a real API response.

What this assumes: Only files you select are read, and only as text: anything with a known binary extension is skipped without being opened, as is anything over the size limit or matching an ignore segment. The scan stops at 3,000 files or 20 MB, whichever comes first, and says so above rather than silently truncating the total. Directories are grouped two levels deep, so app/(marketing) is one row rather than a hundred. A browser folder picker cannot see your .gitignore, so untracked build output is included unless you name it in the ignore field.

The number that matters when you point a coding agent at a repository is not the line count, it is how many context windows the repository is. A codebase that is 1.4 windows and one that is 14 need completely different strategies: the first can be summarised whole in a couple of passes, the second can only ever be searched. The ratio above is that answer, and it changes the moment someone commits a fixture directory.

Which is why the per-directory map is the useful part rather than the total. In almost every repository the distribution is brutally uneven: a lock file, a generated client, a folder of test fixtures or a vendored dependency holds a large share of the tokens and none of the meaning. The default ignore list handles the obvious cases, but the "files worth excluding first" table is there because every codebase has its own particular monster, and finding it is worth more than any amount of prompt tuning.

Two honest limits. First, a browser cannot read your .gitignore — the folder picker sees the working directory as it is on disk, build output and all, so the first scan of a project you have run locally will be dominated by things git never tracked. Add them to the ignore field and rescan. Second, a total like this describes the cost of the naive approach — pasting everything — which is not what a well-built agent does. Retrieval, symbol outlines and file-level search exist precisely so that the number above never has to be paid. Use it as the ceiling to design against, not as a bill.

Repository Token Estimator · Multigrid