Skip to content

Glossary

Time to first token (TTFT)

How long a model takes to start answering, as opposed to how fast it writes once it has started.

Two numbers describe speed and they move independently: the wait before the first word, and the rate of words after it. A model that starts instantly and writes slowly feels quicker than one that thinks for four seconds and then floods the screen, even when the second finishes first.

The wait grows with the input, because the whole prompt has to be read before anything can be produced. This is where prompt caching pays twice — the cached prefix is cheaper and it is also already processed, so the answer starts sooner.

For an agent the arithmetic inverts. Nobody is watching a subagent stream, and a chain of twenty calls pays the startup wait twenty times, so total time matters and the feel of the first word does not.