Korva is a scroll of the internet’s best ideas.
AI models aren't commodities
Models differ
Everyone expected the foundation models to be pure commodities. They have different training methods and different skills. Anthropic is very good at anything that has to do with private equity and finance, and Google is very good at ingesting PDFs. Many people use routers that switch between models, which makes them sound like commodities, but the differentiation is critical intellectual property.
Building an ecosystem
Many companies have come after the coding franchise, and Anthropic has stayed ahead. It is also building a whole ecosystem of products around the API: the SDK, Claude for Co-work, an orchestration layer, and a harness, which is the software around the API that gets the most out of the model.
The AWS parallel
AWS followed the same path in 2013, when people thought of it as servers in a warehouse. AWS saw a new way of doing computing and invented products before anyone else, and those products slowly built lock-in.
How do models end up with different skills?
Each lab chooses its own mix of training data and its own post-training methods. After a model learns from broad text, labs refine it with human feedback and with reinforcement learning on tasks they care about, such as writing code or reading long documents. Those choices shape what the model does best. OpenAI described the human-feedback approach in its 2022 InstructGPT paper.
What does a harness do for a model?
A harness is the software loop around a model: it assembles the prompt, offers the model tools, runs the tools it asks for, and manages what stays in the model's memory. Two products using the same model can perform very differently because of this layer. Coding assistants depend on it to read files and run tests. Context management, which decides what to keep and what to summarize, is one of its hardest parts.
How do model routers choose between models?
A router is a small program, sometimes a small model itself, that looks at each request and sends it to a model suited to its difficulty and cost. Simple requests go to a cheap, fast model, and hard ones go to a stronger, pricier one. Routing is common because prices between models differ by a factor of ten or more. The router can also fall back to another provider when one is down.
Why does cloud lock-in build slowly?
Amazon launched its storage service S3 in March 2006 and its EC2 compute service later that year. As customers built on more of its services, their data and their code became tied to its specific tools. Moving data out also costs money, through fees for data leaving a provider's network. This tie is called data gravity, a term coined by Dave McCrory in 2010.
More from Invest Like the Best with Patrick O'Shaughnessy
