LLM Speed and Latency 2026: Tokens per Second and Time to First Token for Claude, GPT, Mistral and Gemini

Users forgive an AI assistant a mediocre sentence sooner than a blank screen. Speed decides whether a voice agent sounds natural, whether a coding agent keeps a developer in flow and whether a chatbot is used at all. In 2026 latency has a new main cause: thinking. Reasoning models now spend seconds or minutes before they write a word, and providers sell speed as a paid option. We compared the benchmark data of Artificial Analysis for the current models of Anthropic, OpenAI, Mistral and Google, and the fast modes Anthropic and OpenAI now sell.

Three numbers that define speed

"Fast" means three different things, and each matters for a different application:

• Time to first token (TTFT) is the wait until the response starts. For reasoning models the useful variant is time to first answer token, which includes the thinking phase. It decides whether a chat or voice interface feels alive.
• Output speed is the number of tokens per second after the first one. It decides how long a long answer, a report or generated code takes.
• End-to-end response time is the whole thing: input processing, thinking and answer. For agents that make dozens of calls in a row, this is the number that adds up.

Artificial Analysis measures all three about eight times a day with a 10,000-token prompt from a Google Cloud server in the US and reports the median of the past 72 hours. Users in Europe add network latency, and longer prompts raise the time to first token. The numbers below were read on 25 October 2026.

Output speed: small models write two to four times faster

The ranking follows model size. Gemini 3.5 Flash-Lite writes 414 tokens per second, Claude Haiku 5.5 181, Mistral Small 4 178 and Mistral Medium 3.5 162. The frontier models are in a different league: Claude Sonnet 5.5 106, Claude Opus 5.5 80, Claude Fable 5.1 58 and OpenAI's GPT-6 Astra and GPT-6.1 Sol 53 to 57. For a 1,000-token answer, that is about 5.5 seconds of writing with Haiku 5.5, 12.5 seconds with Opus 5.5 and 19 seconds with GPT-6.1 Sol, on top of any thinking time.

Output speed barely depends on reasoning effort: Opus 5.5 writes 80 to 96 tokens/s at every level, GPT-6.1 Sol 51 to 59. What changes with effort is how long the model thinks before it starts writing.

The real latency is thinking

All four providers now let you choose how much a model reasons (Anthropic and OpenAI via an effort parameter, Mistral and Google via separate reasoning models or thinking budgets). The effect on waiting time is enormous. Claude Opus 5.5 needs 6 seconds to its first answer token at low effort, 16 at medium, 30 at high, 110 at extra-high and 653 at max, more than ten minutes. GPT-6.1 Sol goes from 2 to 236 seconds. Claude Sonnet 5.5 answers in under 1.5 seconds at low and medium effort, which makes it the quickest-starting model of the group.

What the extra seconds buy

More thinking does raise quality, but with sharply falling returns. On the Artificial Analysis Intelligence Index, Opus 5.5 gains 9 points from low to medium effort for about 9 extra seconds end to end. From high to max it gains 4 points for more than 10 extra minutes. GPT-6.1 Sol gains 6 points from low to medium and only 4 more across the rest of the range. Sonnet 5.5 at extra-high effort (52 points, 32 seconds) scores about as high as GPT-6.1 Sol at max (52 points, 245 seconds).

Effort also sets the bill

Thinking tokens are billed as output tokens, so latency and cost rise together. Running the whole Intelligence Index costs per task $0.55 with Opus 5.5 at low effort and $5.98 at max, eleven times more. Sonnet 5.5 rises from $0.35 to $5.46. GPT-6.1 Sol stays cheaper per task ($0.13 to $0.72), and Haiku 5.5 is in another category at 2 to 21 cents.

Buying speed: fast modes and special hardware

• Anthropic sells a fast mode for Claude Opus 5.5: the same model on a faster inference configuration, with up to 2.5 times higher output speed. It is enabled per request with speed: "fast" and costs $8 input and $40 output per million tokens, twice the standard $4 and $20. It speeds up output, not time to first token, has its own rate limit, and does not work with the Batch API or a Priority Tier commitment. On Opus 5 and Opus 4.8 it is still a research preview.
• OpenAI renamed its priority processing to Fast mode on 30 July 2026. It is pay-as-you-go at higher prices and, according to OpenAI, up to 2.5 times faster for its frontier model, with a tokens-per-second floor per model. Localised versions of OpenAI's page show different prices, so check the current figures before budgeting.
• Mistral went the hardware route: since February 2025, Le Chat's Flash Answers run on Cerebras chips, which Cerebras states at more than 1,100 tokens per second for Mistral Large 2. In the API, Mistral's small and medium models are among the fastest of any provider.
• Manus is an agent product, not a model API, and publishes no token speeds. Its speed is measured in minutes per task, which depends on the models underneath and the number of steps.

Latency budgets by use case

Three engineering habits matter more than the model choice. Stream every user-facing answer, so the first words appear while the rest is written. Cache long system prompts and documents (see prompt caching), which cuts time to first token for long contexts. And route requests: a small, fast model answers the simple majority and hands hard cases to a frontier model (see model routing).

What this means for companies in Croatia and the DACH region

Most European companies call US-hosted model endpoints, so every request crosses the Atlantic, and EU data residency options can add their own price and capacity limits. That makes the model-side choices more important: an assistant that defaults to max effort feels broken, however smart it is. Measure latency from your own servers with your own prompts, set reasoning effort per task, and decide where speed is worth paying for. A support bot in Zagreb or Vienna needs a quick first token; a monthly risk report can think for ten minutes.

Frequently asked questions

Which AI model is the fastest in 2026?

In the Artificial Analysis benchmark (October 2026), Gemini 3.5 Flash-Lite leads with 414 output tokens per second. Among Anthropic, OpenAI and Mistral models, Claude Haiku 5.5 (173 to 234 tokens/s depending on effort), Mistral Small 4 (178) and Mistral Medium 3.5 (162) are fastest. Frontier models are slower: Claude Opus 5.5 writes 80 to 96 tokens/s, GPT-6.1 Sol 51 to 59.

What is the difference between time to first token and output speed?

Time to first token is how long you wait before anything appears; for reasoning models the useful measure is time to first answer token, which includes the thinking. Output speed is how many tokens per second arrive after that. A chat interface feels responsive when the first token is quick; a long report finishes sooner when output speed is high.

How much does reasoning effort slow a model down?

A lot. Claude Opus 5.5 needs about 6 seconds to its first answer token at low effort, 16 at medium, 30 at high, 110 at extra-high and 653 at max. GPT-6.1 Sol goes from 2 to 236 seconds. Intelligence scores rise by 10 to 16 points across that range, but most of the gain comes in the first steps.

Is Claude fast mode worth paying for?

Fast mode runs Claude Opus 5.5 with up to 2.5 times higher output speed at twice the standard price ($8 input and $40 output per million tokens instead of $4 and $20). It speeds up output, not time to first token. It pays off where a person waits for a long answer, such as coding agents and interactive analysis, and not in background jobs, where the Batch API halves the price instead.