Korva is a scroll of the internet’s best ideas.
Open models now handle 90% of tasks
Picking the right model
Glean uses this for cost control. As people finish tasks on its platform, it picks the right model for each one, and when an open source model will produce a high-quality answer and the customer is fine with it, it uses one.
Open source has only just reached this point. Within about three months of frontier capabilities, GLM 5.2 was the first time Glean's own team felt comfortable running the majority of its workloads on an open model, and that happened within roughly the last month.
Why cost drives it
The drive toward open source is cost. Companies set annual AI budgets and run past them within a month or two, and good open models arrived at the same moment. Fear of data leaving a company's control has faded, apart from businesses required to keep all inferencing inside their own private data centers.
The Chinese model question
The one question left is whether a company is comfortable with a Chinese model. Running it on premises means nothing is shared back, yet worries linger about a hidden back door, and about competitors using the choice against you if it became known. Glean has been telling customers that the majority of enterprise workloads will run on open source models in three years.
Why do open models trail frontier ones by months?
The labs that build frontier models spend heavily on data, training, and fine-tuning, and much of that stays private. Open developers catch up through published research, and through training smaller models on the outputs of larger ones, a method called distillation. Epoch AI estimated in 2024 that open-weight models trailed the best closed models by about a year on benchmarks. The gap is measured differently by different groups and has narrowed over time.
How does software pick a model for each task?
A router is a small program that estimates how hard a request is and sends easy ones to cheaper models. Researchers at LMSYS published a router called RouteLLM in 2024 and reported cutting costs by more than 85 percent on some benchmarks while keeping about 95 percent of GPT-4's quality. The savings come from the fact that most requests do not need the strongest model. Hard requests are still sent to the most capable one.
Why do some firms keep inference in private data centers?
Banks, hospitals, and government agencies often operate under rules about where data may be stored and who may process it. Running a model on premises means requests never leave hardware the organization controls. Open-weight models make this possible because the weights can be downloaded and run locally. Hosted models from outside providers usually cannot be run that way.
More from 20VC with Harry Stebbings
