Korva is a scroll of the internet’s best ideas.
Token-maxing lets you live in 2028
What Costs So Much
The labs offering ChatGPT and Claude are still constrained on cost and on how much compute they want to hand out. Using everything the models can do means running Hermes Agent or OpenClaw and tuning it all the way up, loading 800,000 or a million tokens into any given request.
How It Runs
A retrieval layer decides which million tokens the agent keeps in its head for a given task. Every interaction then runs at full strength, the equivalent of a 150 IQ, and the difference is felt almost immediately. It costs a crazy amount.
Why It Pays
For a CEO or a founder the spending makes sense, and it takes giving yourself permission to token max. The result is living in 2028 today.
How does a retrieval layer choose tokens?
Retrieval-augmented generation, described in a 2020 paper from Facebook AI Research, stores documents outside the model and searches them at question time. The search ranks stored passages by similarity to the task, often by comparing numerical embeddings. The top passages are placed in the prompt, so the model sees only what the search judged relevant. The approach lets a model draw on far more text than fits in a single prompt.
Why do longer prompts cost more?
Providers bill per token, for both the text going in and the text coming out. Inside a transformer model, self-attention compares every token with every other token, so the compute for a long input grows roughly with the square of its length. Anthropic and OpenAI both offer prompt caching, which charges less for input that repeats across requests. Per-token prices for input are typically lower than for output.
More from The a16z Show
