Korva is a scroll of the internet’s best ideas.
xAI releases Grok 4.6 and Grokbot
Grok 4.6 is a model of about 1.5 trillion parameters. A larger model, Grok 4.7, is due in a few weeks.
On a chart of quality against cost, with the cost axis inverted so that up and to the right is better, Grok 4.6 sits on the outer edge of the frontier. An evaluation from Databricks, which has raised money at a $190 billion valuation, puts it well ahead of Fable 5, with higher quality at a slightly lower price.
DHH, the creator of Ruby on Rails and a co-founder of 37signals, posted something really positive about it. A Mercor evaluation was cited as well.
Why do model charts use a Pareto frontier?
A Pareto frontier marks the points where nothing else beats a model on both measures at once. For models, that means no rival has both higher quality and a lower price. Economist Vilfredo Pareto introduced the underlying idea of efficiency in the early 1900s. Inverting the cost axis simply puts the best models in the upper right corner.
Why does parameter count matter for models?
Researchers at OpenAI reported in 2020 that a language model's error falls smoothly and predictably as parameters, data, and computing power grow. DeepMind's 2022 Chinchilla paper then showed that training data should grow in step with model size. A model with too many parameters for its training data wastes compute. Parameters are the adjustable numbers learned during training.
How can a huge model still run cheaply?
Mixture-of-experts designs split a model into many specialist sub-networks and activate only a few for each token. Mistral's Mixtral 8x7B, released in 2023, used about 13 billion of its roughly 47 billion parameters on each token. Compute per answer tracks the active parameters, not the total. Memory still has to hold the whole model.
More from All-In with Chamath, Jason, Sacks & Friedberg
