Conversation
Deterministic — the same scenario and count always produce the same conversation.
Context budget
Before / after
Result
Why? (explain())
Benchmarks
token-budget is a context-management library, not a raw tokenizer. Raw tokenizer speed is not its primary goal — see the honest numbers below, not a claim that it's the fastest way to count tokens.
Raw tokenization benchmark (reference)
Context-management benchmark
Runs live in this browser tab, comparing token-budget against a naive DIY shift()-and-recompute loop, using the
same simple length-based token counter for both — so this isolates buffer/eviction machinery cost, not tokenizer
choice. Kept small enough to stay responsive; the full 50,000-message benchmark (including a real LangChain trimMessages
comparison) is documented below with the exact command to reproduce it locally.
Published reference numbers (50,000 messages)
Run the full benchmark locally
git clone https://github.com/shivam039/token-budget.git
cd token-budget
npm install && npm run build
npm run bench
Full methodology, environment, and every number: docs/benchmarks.md.