Claude Turns the Same TypeScript Into 73% More Tokens. That Still Isn't the Bill.

PlayCode's tokenizer analysis opens with a number that deserves attention: its 2,888-character TypeScript fixture became 681 tokens under OpenAI's o200k tokenizer and 1,178 under Claude's newer tokenizer. That is 73% more billable input units before comparing the providers' per-token rates.

My read is that the post exposes a real blind spot, then gets close to the more useful conclusion near the end: the real unit is not dollars per million native tokens. It is dollars per successful task on your workload. Tokenizer efficiency changes the meter, but output length, reasoning, caching, retries, latency, and whether the result actually works determine the bill that matters.

Answer Snapshot

Question	My read
What happened?	PlayCode counted 16 fixed content samples with provider token-counting tools and found that Claude's newer tokenizer produced 73% more tokens than GPT's o200k on its TypeScript fixture.
Why it matters	A token is native to a model family, so equal prices per million tokens do not necessarily buy equal amounts of code or prose.
Who benefits?	Teams running coding agents, large prompts, tool schemas, JSON, or repeated repository context can make better routing and budgeting decisions.
What is the catch?	One tokenizer fixture does not measure model quality, verbosity, thinking tokens, cache behavior, retries, or successful task completion.
My thesis	Measure native token counts to catch hidden input inflation, then choose models by cost per accepted result rather than by either token count or sticker price alone.

The 73% Gap Is Narrow, But Useful

PlayCode says it tested 16 byte-identical fixtures spanning English prose, HTML, JavaScript, Python, TypeScript, Rust, JSON tool traffic, Chinese text, symbols, and an agent system prompt. It used Anthropic's token-counting endpoint for Claude, o200k through tiktoken plus live usage checks for the GPT models, and provider counters for Gemini and Grok. That is a much better method than estimating tokens from characters.

The TypeScript result is also not presented as universal. In PlayCode's table, Claude's newer tokenizer ranged from 1.50 times GPT's count for Python to 1.73 times for TypeScript across the code fixtures. English prose was 1.40 times. I like that boundary because it keeps a striking headline from becoming a fake law of nature.

Anthropic's own token-counting documentation independently supports the same-vendor part of the story. It says newer Opus-family models, Fable 5, Mythos-family models, and Sonnet 5 use a newer tokenizer that produces approximately 30% more tokens than earlier models for the same input, with the exact increase depending on content and workload shape. Anthropic explicitly tells developers to recount prompts for the model they plan to use.

Native Tokens Are Different Units

The conceptual mistake is treating every provider's token as if it were a standardized gram. It is closer to buying material by the box when each supplier chooses a different box size. A price per box is useful within one catalog. It becomes misleading across catalogs unless I also know how many boxes the same job needs.

Artificial Analysis makes this distinction explicit: native tokens are produced by a model's own tokenizer, while its cross-model performance measurements use OpenAI-token equivalents as a common ruler. Its cost-per-task metric then multiplies input, cached-input, and output prices by the tokens consumed across a workload. That is the normalization I want when comparing systems rather than rate cards.

The source is especially relevant to coding agents because their inputs are not just a user's short request. They can include long system instructions, tool definitions, repository files, generated patches, test output, and an expanding conversation. If one tokenizer splits that repeated payload more aggressively, the difference compounds across turns.

But this is where I would resist the most dramatic interpretation. Token count is not capability. A model that consumes more input tokens may still complete the task with fewer turns, produce a better patch, or require less review. A lean tokenizer can coexist with verbose reasoning or failed attempts. The cheapest input meter does not automatically build the cheapest working feature.

The Missing Variable Is Completion

The strongest external evidence I found pushes beyond tokenization. A Microsoft Research study evaluated eight reasoning models across nine tasks and found that the lower listed price produced the higher total cost in 21.8% of model-pair comparisons. The researchers traced much of the reversal to differences in thinking-token use, and they observed substantial cost variation even across repeated runs of the same query.

That does not weaken PlayCode's measurement. It places it in the right stack. Native input tokenization is one multiplier. Reasoning and visible output are others. Caching can discount repeated prefixes. Tool calls, failed tests, and repair loops add more work. Finally, the result has to pass whatever acceptance test made the call worth buying.

Public discussion around cost-per-task makes the same correction from another angle. In a recent r/LLMDevs thread, one skeptical response argued that per-token pricing is a consumption rate, while benchmarks and users still have to verify useful output. Another recurring point was that review and recovery steps can prevent one model error from cascading through an agent workflow. I find that critique persuasive: retries are a cost, but a well-designed verifier is also how an agent turns probabilistic work into an accepted result.

I Would Benchmark The Workload In Two Passes

First, I would run a representative frozen input set through each provider's counting endpoint. It should include the real system prompt, tool schemas, common source files, JSON responses, and at least one long conversation state. I would separate cacheable and changing content instead of blending everything into one average. This pass reveals the tokenizer multiplier PlayCode found.

Second, I would run the actual task several times and record the complete usage response, elapsed time, tool calls, retries, test result, and human review outcome. The denominator should be accepted tasks, not attempted tasks. For coding work, an accepted task might mean the patch passes tests, satisfies a review rubric, and does not create a follow-up repair ticket.

I would also use a distribution rather than one triumphant run. Microsoft Research found repeated-query variability in thinking-token consumption, and agent workflows add even more sources of variance. Median cost is useful; a high-percentile cost is what protects a budget.

My Bottom Line

PlayCode is right about the category error. Dollars per million native tokens are not directly comparable when each model decides what a token is. Its TypeScript fixture makes that abstraction concrete, and Anthropic's documentation confirms that a tokenizer change can materially increase counts even within one model family.

I just would not stop at the 73%. For a coding agent, the purchasing question is not which tokenizer makes the repository smallest. It is which model-and-workflow combination produces an accepted change at the best cost, speed, and reliability. Count the code first. Then count the whole job.