Korva is a scroll of the internet’s best ideas.

Jagged capability

Model capability is jagged: excellent at some things and failing at others, in a pattern very different from the smooth arc of human skill that people understand intuitively. A builder has to know where the edges are and build around them, and the edges move as new capabilities arrive.

Cursor and Sierra

Cursor's team understood this from the start. They built a product that translated between developers and that jagged capability, and they kept iterating as the edge changed. Sierra works the same way: an application company on the surface that is doing real AI work, building agents, and sitting very close to the models.

Blurring job titles

Job titles blur under this. The people who do great have three things: they understand customer problems, they have taste, and they are curious about the jagged edge of AI capabilities. Whether the title reads engineer, product manager, or designer matters little.

Where does the jagged frontier idea come from?

A 2023 study by researchers at Harvard Business School and Boston Consulting Group gave 758 consultants tasks with and without GPT-4. On tasks inside the model's abilities, consultants worked faster and produced higher quality results. On a task chosen to fall outside its abilities, consultants using the model were 19 percentage points less likely to reach a correct answer. The researchers called the uneven boundary the jagged technological frontier.

Why do models fail at simple tasks?

Language models read text as tokens, which are chunks of characters, not as individual letters. Asking one to count the letters in a word therefore asks about units it never directly sees. The same model can still write a strong essay on the word's history. Tokenization is one reason capability varies so much from task to task.

Why is human skill smooth by comparison?

The psychologist Charles Spearman reported in 1904 that people's scores on different mental tests are positively correlated. He named the shared factor general intelligence. A person who handles calculus can almost always handle arithmetic, so skills tend to rise together. Models are trained differently and do not share that tidy ordering.

More from Invest Like the Best with Patrick O'Shaughnessy

Tech & AIRobots lack internet-scale dataLanguage models bootstrapped on the internet. Robots have no equivalent, so the companies working on them start by collecting a small set of the highest-value data.
Enjoyed it?