AI in Croatian and German 2026: Multilingual LLM Quality and the Token Tax

Almost every AI price list, benchmark and speed chart is measured in English. A company in Zagreb, Graz or Munich works in Croatian or German, and two things change there: the quality of answers and the number of tokens the same text needs. The second is easy to overlook, because providers charge the same price per token in every language. We measured it ourselves: 488 identical news passages in 12 languages, run through 7 tokenizers of OpenAI, Mistral, Google, Meta, Alibaba and DeepSeek. Then we compared what Anthropic, OpenAI and independent researchers report about quality outside English.

Why language changes the bill

Models do not read words or letters but tokens, pieces of text from a fixed vocabulary that the tokenizer learned mostly from English data. Frequent English words are a single token. Croatian words with their many case endings ("poduzećima", "odgovornostima") are split into several pieces. The result: the same content needs more tokens, and every extra token is billed, takes up room in the context window and has to be generated one by one.

Our test used the FLORES-200 passages from Meta's Belebele dataset, professionally translated news texts that say exactly the same thing in each language. The Croatian version has as many characters as the English one (229,000 vs. 232,000), yet with OpenAI's o200k tokenizer it needs 69,600 tokens instead of 46,900. English fits about 4.9 characters into one token, Croatian 3.3, German 4.4.

The newest tokenizers have narrowed the gap considerably. OpenAI's older cl100k (GPT-4) needed 1.85 times as many tokens for Croatian; o200k needs 1.48. Mistral's Tekken (1.50) and Google's Gemma 3 tokenizer (1.47) are on the same level. Llama 3, Qwen3 and DeepSeek V3 still charge Croatian 1.78 to 1.84 times English. German, with its long compound words, is cheaper at 1.30 to 1.58.

Twelve languages, seven tokenizers

The pattern holds across Europe: the big Western European languages pay the least, Central and Eastern European languages more, and Cyrillic the most. Serbian in Cyrillic needs 1.78 times English with o200k but 2.92 with cl100k; Ukrainian up to 3.0. Croatian sits in the middle of the Slavic group, slightly cheaper than Slovenian, Czech and Polish.

Anthropic does not publish Claude's tokenizer, so Claude is missing from our table. The API's free token counting endpoint gives the exact number for your own texts, and that test is worth doing before you budget a Croatian or German application.

Quality: German is almost on par with English

Anthropic publishes how Claude performs on MMLU translated into 14 languages by professional translators, relative to English. Claude Sonnet 4.5 with extended thinking reaches 97.0% of its English score in German, 97.5% in French and 98.2% in Spanish. The smaller Claude Haiku 4.5 keeps 94.3% in German. The gap grows for languages with little training data: Swahili 91.1% and 78.3%, Yoruba 79.7% and 52.7%.

OpenAI's numbers on the same benchmark tell the same story. Its o3 model (high reasoning) scores 90.5% accuracy in German, 90.6% in French and 91.1% in Spanish; GPT-4.1 scores 85.5% in German. Smaller models lose more: GPT-4.1 nano drops to 72.2%. The lesson for German: a frontier model is close to its English level, a small model noticeably less so.

Croatian: few official numbers, encouraging research

Neither Anthropic nor OpenAI includes Croatian in its published language tests, and Mistral publishes no per-language scores. The best independent source is the CLASSLA team (Kuzman Pungeršek, Ljubešić et al., revised February 2026), which tested LLMs on Croatian, Serbian, Bosnian, Slovenian and English data for sentiment, news topics, parliamentary topics and web genres. Their results:

• The gap to English is small. LLMs lost 2 to 7 macro-F1 points on sentiment and 3 to 10 points on the harder 22-label topic task in South Slavic languages compared with English.
• Closed models lead. GPT-5, GPT-4o and Gemini 2.5 Flash ranked best on South Slavic data; Mistral Medium 3.1 and Gemma 3 led the rest. Claude was not part of the study.
• Zero-shot beats training for broad tasks. On sentiment, most LLMs matched or outperformed a BERT-like model fine-tuned on a large annotated dataset. For narrow label schemes, fine-tuned models still won, and they were three to seven times faster.

Typical weak spots in Croatian and German

• Mixed variants. Croatian, Serbian and Bosnian share much of their training data, and models sometimes slip Serbian words or forms into Croatian text. Name the variant ("standard Croatian") in the system prompt and give a short terminology list.
• Formal address. "Vi" or "ti", "Sie" or "du": models change register within a conversation unless told which one to use.
• Grammar in small models. Case endings and agreement suffer first when you choose a small, cheap model. Test the model you will actually run, not the flagship from the benchmark.
• Local knowledge. Croatian laws, institutions and tax rules are thinly represented in training data. Use retrieval from your own documents (see RAG) instead of relying on the model's memory.

Five rules for multilingual AI projects

What this means for companies in Croatia and the DACH region

For German-speaking companies, language is no longer a reason to wait: frontier models work at almost their English level, and the token premium of about 30% is a known cost item. Croatian companies pay more, about 50% with the best current tokenizers, and have to test quality themselves, because the large providers publish no Croatian numbers. The research shows the gap is smaller than feared. Budget with your own token counts, evaluate in Croatian with Croatian speakers, and use the savings tools (caching, batch, routing) that make the premium irrelevant in practice.

Frequently asked questions

Does AI cost more in Croatian than in English?

Yes, with token-based pricing. In our measurement on 488 identical passages, Croatian needed 1.47 to 1.85 times as many tokens as English depending on the tokenizer: 1.48 with OpenAI's o200k_base (GPT-4o to GPT-5), 1.50 with Mistral's Tekken and 1.47 with Gemma 3. German needed 1.30 to 1.58 times as many. Since price per token is the same in every language, the same work costs that much more.

How good are Claude and GPT in German?

Very close to English. On a professionally translated MMLU test, Anthropic reports German at 97.0% of English performance for Claude Sonnet 4.5 and 94.3% for Claude Haiku 4.5. OpenAI's o3 (high) scores 90.5% accuracy in German, about the same as in Spanish or French.

How good are large language models in Croatian?

Good, but less well documented. Anthropic and OpenAI do not publish Croatian scores. A 2026 study by the CLASSLA team found that LLMs lose only 2 to 10 macro-F1 points on South Slavic text classification compared with English, and that GPT-5, GPT-4o and Gemini 2.5 Flash were the best models; zero-shot LLMs matched or beat fine-tuned BERT-like models on sentiment.

Should I write prompts in English or in Croatian?

Long, stable instructions in English are cheaper, since English uses the fewest tokens, and models follow them reliably. State the output language explicitly in the system prompt, include examples in the target language, and evaluate the answers with native speakers. Customer-facing text and terminology lists should stay in the target language.