Rebuilding a small model's vocabulary cut one team's serving cost by 77%

Dharma-AI runs a Portuguese essay grader built on Qwen3-4B. Its tokenizer was built for many languages, so the same JSON wrapper and grading phrases took many tokens on every reply. The team rebuilt the vocabulary with a method it calls VocabX, picking new tokens by how much they save on cost, where a token the model writes cost 7.4 times a token it reads in their setup. Average reply length fell from 350 to 101 tokens, measured cost fell 77.0%, and the grader's quality score rose from 0.703 to 0.810, with the same base model, data and training recipe. These are early results from a paper still under review, measured on one RTX 5090 with vLLM over 500 requests, and the numbers may change.