Claude's Tokenizer Counts 30% More Tokens: What That Actually Does to Your Bill

Since Claude Opus 4.7, Anthropic uses a new tokenizer that splits the same text into roughly 30% more tokens. The advertised price per million tokens has not moved. Your bill has. It's a real price increase that doesn't look like one, because it doesn't run through the rate, it runs through the unit of measurement.

Put differently: a million tokens no longer buys the same amount of text. The meter spins faster for the same work.

Anthropic isn't hiding it. It's written plainly in the docs. But nobody emails you when a footnote reshapes your monthly budget. Here's what it actually changes.

The official number: about 30%, in a 0 to 35% range

Anthropic's pricing docs are explicit. Recent models "use a newer tokenizer that contributes to their improved performance on a wide range of tasks. This tokenizer produces approximately 30% more tokens for the same text. The exact increase depends on the content and workload shape."

The migration guide gives the full range: the new tokenizer "may use roughly 1x to 1.35x as many tokens when processing text," so up to about 35% more depending on content.

Translated into actual text volume, you get the figure that lands hardest: one million tokens holds around 555,000 words under the new tokenizer versus 750,000 under the old one. Same 1M context window, nearly 200,000 fewer words fit inside it.

The stated tradeoff is genuine: this tokenizer is part of why the models perform better. The problem isn't that it exists. It's that you're billed for it while the rate stays frozen.

Which models are affected, which are spared

The fault line is clean, and it runs at Opus 4.7. Everything released after uses the new tokenizer. Everything before keeps the old one.

Model	Tokenizer	Sticker price (input / output)	Estimated effective increase
Claude Opus 5	New	$5 / $25 per MTok	+12 to +27% measured (OpenRouter)
Claude Opus 4.8	New	$5 / $25 per MTok	+12 to +27% measured (OpenRouter)
Claude Opus 4.7	New	$5 / $25 per MTok	+12 to +27% measured (OpenRouter)
Claude Opus 4.6	Old	$5 / $25 per MTok	Baseline, 0%
Claude Sonnet 5	New	$2 / $10 through Aug 31, 2026, then $3 / $15	Neutral until Aug 31, then +0 to 35%
Claude Sonnet 4.6	Old	$3 / $15 per MTok	Baseline, 0%
Claude Fable 5	New	$10 / $50 per MTok	2x price and 1.3x tokens vs Opus 4.6, so roughly 2.6x per unit of text
Claude Mythos 5	New	$10 / $50 per MTok	Same as Fable 5
Claude Haiku 4.5	Old	$1 / $5 per MTok	Baseline, 0%

Sonnet 5 deserves a pause. Anthropic launched it at an introductory rate of $2 / $10 running through August 31, 2026, moving to $3 / $15 after that. The intro rate absorbs the tokenizer inflation and makes the migration roughly cost-neutral. On September 1, the price snaps back to Sonnet 4.6's rate ($3 / $15), but with a tokenizer that produces about 30% more tokens than Sonnet 4.6 did. Same sticker, same billing model, heavier invoice. The increase has a date, and it's already on the calendar.

What it looks like on a real invoice

Take a concrete workload running on Opus 4.6: 200 million input tokens and 20 million output tokens per month. At Opus rates of $5 / $25 per million, that's $1,000 of input plus $500 of output, so $1,500.

You migrate to Opus 4.8. Same code, same prompts, same traffic, same sticker price. At a 1.30 ratio, those same texts become 260M input tokens and 26M output tokens: $1,300 plus $650, so $1,950. You pay $450 more per month for exactly the same work, and not a single price line changed.

Except reality is more nuanced than that raw math, and that's the interesting part.

OpenRouter analyzed over one million production requests by isolating a "switcher cohort": users whose top model was Opus 4.6 before 4.7 shipped, and who then moved to 4.7. A clean before-and-after on the same user base.

In native tokens, the 4.7 tokenizer produces 32 to 34% more tokens than 4.6 on production-scale prompts (10K tokens and up), and up to 42 to 45% more on small prompts.

In real cost, once caching is factored in:

Under 2K tokens: -1.6% (it got cheaper)
2K to 10K: +27.2%
10K to 25K: +25.2%
25K to 50K: +21.3%
50K to 128K: +11.9%
128K and up: +15.3%

The gap between 32-45% more tokens and 12-27% more cost has a name: prompt caching. On prompts of 128K and above, 93% of the extra tokens land in cache, and a cache hit bills at 10% of the standard input price. The extra tokens are real, they're just billed at ten cents on the dollar. If you don't cache, you eat the full 30%.

Your overhead depends heavily on what you send

The 30% average hides enormous spread by content type. A measurement run on real Claude Code content lands at a weighted average ratio of 1.325x, with this distribution:

CLAUDE.md file: 1.445x
User prompt: 1.373x
Blog post in Markdown: 1.368x
Git commit log: 1.344x
Terminal output: 1.291x
Python stack trace: 1.250x
Code diff: 1.212x

On synthetic content the range widens further: 1.47x for technical documentation, 1.20x for English prose, and only 1.01 to 1.13x for JSON, CSV, or Chinese. Simon Willison independently measures 1.42x on English, 1.33x on Spanish, 1.27x on Python, and 1.01x on Mandarin.

So: if your workload is structured JSON, you'll barely notice. If it's technical docs, Markdown, and fat context files, you're at the top of the range. The single worst profile is precisely a coding agent with a large CLAUDE.md.

The Register, which covered the story on July 14, 2026, adds the cross-vendor angle. On a 2,888-character TypeScript file, Claude's new tokenizer emits 1.73x more tokens than GPT-5.x's and 1.32x more than Claude's old tokenizer. The same piece notes the ratio against GPT-5.x hits 1.58x on Rust, 1.52x on JavaScript, and 1.50x on Python. Comparing sticker prices between Anthropic and OpenAI no longer means much: it isn't the same unit.

How to measure it yourself

Don't trust any percentage in this article, Anthropic's included. Your workload has its own ratio, and the spread between 1.01x and 1.47x is far too wide to settle for an average. Good news: measuring is free.

The /v1/messages/count_tokens endpoint returns the token count under the tokenizer of whichever model you pass in. It's free, rate-limited to 2,000-8,000 requests per minute depending on your tier. Anthropic's own recommended method: count the same request twice, once with your current model and once with the new one, then compare the two input_tokens values.

import anthropic

client = anthropic.Anthropic()

# Load a representative sample of YOUR real workload
with open("real_prompt.txt") as f:
    prompt = f.read()

def count(model: str) -> int:
    resp = client.messages.count_tokens(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return resp.input_tokens

old = count("claude-sonnet-4-6")   # old tokenizer
new = count("claude-opus-4-8")     # new tokenizer

print(f"Old      : {old} tokens")
print(f"New      : {new} tokens")
print(f"Ratio    : {new / old:.3f}x")
print(f"Overhead : {(new / old - 1) * 100:.1f}%")

One curl call, if you just want a ballpark:

curl https://api.anthropic.com/v1/messages/count_tokens \
  --header "x-api-key: $ANTHROPIC_API_KEY" \
  --header "content-type: application/json" \
  --header "anthropic-version: 2023-06-01" \
  --data '{
    "model": "claude-opus-4-8",
    "messages": [{"role": "user", "content": "Your prompt here"}]
  }'

Three things that matter:

Measure on real content, not "Hello world." The ratio is content-dependent. Pull ten prompts straight out of your production logs, not an invented example.

Weight by your actual mix. If 80% of your volume is JSON at 1.05x and 20% is Markdown at 1.37x, your effective ratio is around 1.11x, not 1.30x.

Revisit your max_tokens. The migration guide is direct: you need to give max_tokens additional headroom, including compaction triggers, because the same response text now occupies more tokens. And any code that estimates tokens client-side with a fixed character-to-token ratio needs retesting: it's been wrong since Opus 4.7.

What to take away

The sticker price is no longer a reliable cost signal. Between Opus 4.6 and Opus 4.8 the price is identical ($5 / $25) and the bill still climbs 12 to 27% depending on prompt size. Between Anthropic and OpenAI, comparing per-million rates is comparing a price per kilo to a price per pound.

Three concrete moves, in order:

Measure your real ratio with count_tokens on your actual prompts. It's free and takes ten minutes.
Turn on prompt caching if you haven't. It's the difference between +32% and +12% on large contexts.
If you're on Sonnet 5, put August 31, 2026 in your calendar. The intro rate absorbing the tokenizer ends that day, and your bill mechanically takes a 50% hit on sticker price alone, before the tokenizer even enters the math.

The rest is arbitrage. Old-tokenizer models (Sonnet 4.6, Opus 4.6, Haiku 4.5) are still available and bill at the same rate for fewer tokens. On workloads where you don't need the performance gain, they have become, mechanically, cheaper than the new ones.