AI companies charge you 60% more based on your language, BPE tokens
The Biggest Con of the 21st Century: Tokens
How AI Companies Are Charging You More Without You Even Realizing It
You pay for what you use. That's the deal. Except it's not.
When you use an AI model — GPT-4, Claude, Gemini — you do not pay per word. You pay per token . And that tiny technical detail is quietly costing you, depending on which company you choose, up to 60% more for the exact same request .
Before we get to the money, a crash course. Tokens are not words. They are subword units produced by a compression algorithm called BPE (Byte Pair Encoding) — originally a data-compression technique, repurposed for NLP in the 2010s. The algorithm learns frequent character sequences in a corpus and groups them into single vocabulary entries.
The catch: every AI company trains its own tokenizer on its own corpus with its own vocabulary size. The result is that the same word gets sliced differently depending on who's counting: