Markdown to Plain Text
Strip markdown formatting while keeping the text, with an exact character saving, an estimated token saving and a count of every construct that changed.
Estimated tokens: 184 → 130, a saving of about 54. That is an estimate from your divisor, not a tokeniser's count.
- Characters before
- 736
- Characters after
- 519
- Headings unwrapped
- 2
- Links flattened
- 1
- Images to alt text
- 1
- Emphasis marks removed
- 3
- Code blocks
- 1 kept
- Inline code spans kept
- 1
- List items
- 3
- Block quote lines
- 1
- Table rows
- 3
- Horizontal rules removed
- 1
- HTML tags removed
- 2
- 1 link URL was dropped and only the link text kept. A model cannot cite a URL it was not given.
- 1 image became alt text. Where the alt text was empty, the image is now nothing at all.
Markdown is mostly punctuation, and punctuation tokenises badly. A heading costs a hash and a space; a bold phrase costs four asterisks; a link costs the entire URL, which is often longer than the sentence it sits in. None of that is free, and in a retrieval pipeline where you are pasting forty chunks into every request it is the same tax forty times.
Stripping is not always right
Structure is information. A model reading a table with the columns collapsed into a run of words cannot tell which value belongs to which heading, and a list flattened into a paragraph loses the fact that the items are peers. That is why every removal here is a toggle with a count next to it rather than an opinion: the question is not whether markdown costs tokens — it does — but whether that particular structure was carrying meaning your model needs.
Read the saving carefully
The character count is exact and the token figure is not. Tokenisers do not split text evenly, and formatting characters are exactly the ones they handle least predictably: a run of asterisks and brackets can become several tokens where the same number of ordinary letters becomes one. So the real saving on a markdown-heavy document is often larger than the estimate above, and on a code-heavy one it can be smaller. Use it to decide whether stripping is worth doing at all, and a real tokeniser to decide whether the result fits.